Glossary

GPU cloud glossary. The terms behind price and performance.

What HBM3e, NVLink, SXM, FP4 or a GPU-hour actually mean, and how each one changes what a GPU server costs or how fast it runs. 35 terms, one paragraph each.

35 terms
6 topics
Figures from vendor datasheets

Hardware

Dedicated GPU: A GPU reserved for one customer for the whole rental period: not shared with other tenants, not partitioned into slices and not time-shared. You get its full memory, compute and interconnect bandwidth.
Bare metal server: A physical server rented whole, with no hypervisor between the customer and the hardware, so the customer controls the operating system, the drivers and low-level settings. A dedicated server can also be delivered as a single-tenant virtual machine with the GPUs passed through.
SXM: NVIDIA's socketed GPU module for its HGX boards, used by the H100, H200, B200 and B300. SXM GPUs run at higher power than PCIe cards and connect to each other over NVLink.
OAM: The OCP Accelerator Module, an open GPU module form factor. AMD Instinct MI300X, MI325X, MI350X and MI355X use OAM modules on 8-GPU UBB boards linked by Infinity Fabric.
PCIe GPU card: A GPU on a standard PCIe add-in card, such as the NVIDIA L40S (PCIe 4.0) or RTX PRO 6000 Blackwell Server Edition (PCIe 5.0). Several cards in one server exchange data over PCIe rather than an NVLink mesh, which suits one model per GPU better than tightly coupled multi-GPU training.
HGX: NVIDIA's 8-GPU server baseboard (HGX H100, H200, B200 and B300) on which the eight SXM GPUs share one NVLink domain. Server makers build complete HGX servers around it.
NVL72 (rack-scale): NVIDIA GB200 NVL72 and GB300 NVL72 are liquid-cooled racks in which 72 Blackwell GPUs and Grace CPUs share one NVLink domain. They are sold as whole racks; an HGX server has 8 GPUs.
TDP: Thermal design power, the maximum sustained power a GPU is built to draw, in watts: 350 W for an L40S, 700 W for an H100 SXM, 1,100 W for a B300 and up to 1,400 W for a liquid-cooled AMD MI355X. It decides the power and cooling a data center must provide.

Memory and number formats

HBM, HBM3, HBM3e: High Bandwidth Memory: DRAM stacked next to the GPU die on the same package, giving terabytes per second of bandwidth. The H100 SXM has 80 GB of HBM3 at 3.35 TB/s, the H200 141 GB of HBM3e at 4.8 TB/s, and the B200 and B300 180 GB and 288 GB of HBM3e at 8 TB/s.
GDDR6, GDDR7: Graphics memory soldered around the GPU on PCIe cards: cheaper and slower than HBM. The L40S has 48 GB of GDDR6 at 864 GB/s and the RTX PRO 6000 Blackwell 96 GB of GDDR7 at 1.6 TB/s.
Memory bandwidth: How fast a GPU reads and writes its own memory, in TB/s. LLM token generation (decoding) is usually limited by bandwidth rather than compute, so bandwidth sets tokens per second for a given model.
BF16: Bfloat16, a 16-bit floating-point format with the range of FP32. It is the standard precision for training, and stores a model in 2 bytes per parameter: about 140 GB for a 70B model.
FP8: An 8-bit floating-point format supported by NVIDIA Hopper and Blackwell, Ada Lovelace and AMD CDNA 3 and CDNA 4 GPUs. It halves memory and roughly doubles Tensor throughput compared with BF16, and is widely used for inference and increasingly for training.
FP4: A 4-bit floating-point format run natively by NVIDIA Blackwell (B200, B300, RTX PRO 6000) and AMD CDNA 4 (MI350X, MI355X) GPUs. It stores a model in half a byte per parameter and gives the highest inference throughput, at some cost in accuracy.
Quantization: Storing a model's weights, and sometimes activations, at lower precision (FP8, INT8, FP4 or INT4 such as AWQ or GPTQ) to fit it in less memory and run it faster. 4-bit weights work on any GPU; FP4 compute needs Blackwell or CDNA 4.
KV cache: The memory an LLM keeps for the attention keys and values of every token in flight. It grows with context length and with the number of concurrent requests: a 70B model with grouped-query attention needs about 0.33 MB per token in BF16, roughly 42 GB for one 128K-token sequence.

Compute and interconnect

PFLOPS, dense and sparse: Quadrillions of floating-point operations per second. Vendors quote Tensor throughput both dense and with 2:4 structured sparsity, which doubles the figure; CoreGPU always quotes dense figures, the ones most workloads reach.
Tensor Cores and Matrix Cores: Units inside the GPU that execute matrix multiplications, the core of neural networks, at low precision (FP16, BF16, FP8, FP4). NVIDIA calls them Tensor Cores, AMD Matrix Cores.
NVLink and NVSwitch: NVIDIA's GPU-to-GPU interconnect. On an 8-GPU HGX server, NVSwitch chips connect every GPU to every other: 900 GB/s per GPU with NVLink 4 on Hopper (H100, H200) and 1.8 TB/s with NVLink 5 on Blackwell (B200, B300).
Infinity Fabric: AMD's link between Instinct GPUs on an 8-GPU board: 896 GB/s per GPU on the MI300X and MI325X, 1.08 TB/s on the MI350X and MI355X.
Data, tensor and pipeline parallelism: Ways to spread training or inference over several GPUs: data parallelism copies the model and splits the batch, tensor parallelism splits each layer across GPUs, and pipeline parallelism places groups of layers on different GPUs. Tensor parallelism needs the fastest links, such as NVLink.
NCCL and RCCL: The collective communication libraries (all-reduce, all-gather) that multi-GPU training uses on NVIDIA (NCCL) and AMD (RCCL) GPUs. Their bandwidth tests are the standard check that a new server's interconnect works at full speed.
MIG: Multi-Instance GPU: an NVIDIA feature that splits one GPU into isolated slices with their own memory and compute. Clouds use it to sell fractions of a GPU; a dedicated GPU is never partitioned.

Software

CUDA: NVIDIA's platform for programming its GPUs, with the libraries (cuDNN, cuBLAS, NCCL) that PyTorch, TensorFlow, JAX and most AI software build on.
ROCm and HIP: AMD's open software platform for Instinct GPUs. HIP is its CUDA-like programming interface; PyTorch, JAX, Triton, vLLM and SGLang run on ROCm, and AMD's HIPIFY tools port CUDA code to HIP.
vLLM: An open-source LLM inference server that serves models through an OpenAI-compatible API, with paged attention and continuous batching for high throughput. It runs on NVIDIA and AMD GPUs.

Pricing and billing

GPU-hour: One GPU used for one hour, the billing unit of hourly GPU clouds. A month has about 730 hours, so a monthly price divided by 730 gives its hourly equivalent.
On-demand, spot and reserved pricing: On-demand is billed per hour or second with no commitment; spot or interruptible capacity is cheaper but can be reclaimed at short notice; reserved or committed pricing lowers the rate in exchange for a 1 to 36-month term paid whether the GPUs are used or not.
Monthly GPU billing: One flat price per GPU for each month the server is rented, whatever the GPU does. At CoreGPU the month starts at delivery, renews from a prepaid balance and can be stopped at the end of any month.
Break-even utilisation: The number of hours a month above which a monthly price costs less than paying by the hour. With CoreGPU priced at about 75% of the on-demand median, the break-even is about 540 hours a month, roughly 18 hours a day.
Market median (CoreGPU): The reference CoreGPU prices against: the median public on-demand price of the same GPU at specialist GPU clouds, multiplied by 730 hours. Hyperscalers are excluded; every CoreGPU price is at least 25% below it.
Allocation: A fixed batch of GPUs that CoreGPU puts on sale at once. Stock counts show what is left in the current allocation; a sold-out GPU shows the date of the next one, at the same price per GPU.

Payments

Top-up and bonus credit: Adding US dollars to a CoreGPU balance by sending cryptocurrency. Top-ups run from $100 and earn bonus credit, from +5% at $100 to +30% from $1,000; servers and renewals are paid from the balance.
USDT on TRON (TRC-20): Tether's dollar stablecoin issued on the TRON network. CoreGPU accepts USDT on TRON only: USDT sent on Ethereum, BNB Chain or any other network to a TRON address cannot be credited.
Confirmations: The number of blocks added after a transaction before it is treated as final. A CoreGPU top-up is credited once its network confirms it: under a minute on Solana, 1 to 3 minutes for USDT on TRON, 2 to 5 minutes on Ethereum, 10 to 30 minutes for Bitcoin and about 20 minutes (10 confirmations) for Monero.

Put the terms to work

Compare memory, bandwidth and FP8 or FP4 side by side, then rent the GPU that fits, by the month.

Compare GPUs All GPUs and prices