A dedicated GPU server is a physical server whose GPUs are reserved for a single customer. Nobody else shares, partitions or time-slices them: you get their full memory, their full interconnect bandwidth and root access to the operating system, usually for a fixed monthly price. It is the opposite of renting a slice of a GPU, or an hourly instance on a shared cloud host.
Dedicated, cloud instance, fractional or marketplace
GPU compute is sold in four main ways. They differ in what is actually yours and in how you are billed.
| Model | What is yours | Usual billing | Best for |
|---|---|---|---|
| Dedicated GPU server | Whole GPUs and the server they sit in, with root access | Monthly or longer | Training runs, fine-tuning, 24/7 inference |
| Cloud GPU instance | Whole GPUs passed through to a virtual machine; the host and its network can be shared | Per hour or per second | Elastic workloads, short experiments |
| Fractional GPU | A partition (such as NVIDIA MIG) or a time slice of one GPU | Per hour | Notebooks, small models, development |
| GPU marketplace | A machine run by a third-party host, with variable hardware and uptime | Per hour, often interruptible | Cheap, restartable jobs |
What "dedicated" changes in practice
- Predictable performance. No other tenant competes for the GPUs, their memory bandwidth or the links between them, so a benchmark you run on day one still holds on day thirty.
- The full interconnect. On an 8-GPU NVIDIA HGX or AMD Instinct server, every GPU talks to the others over NVLink or Infinity Fabric. Tensor and pipeline parallelism depend on that bandwidth; a partitioned GPU or a PCIe-only setup cannot provide it.
- Control of the software stack. With root access you choose the OS, the driver, the CUDA or ROCm version and the container runtime, and you can install profilers and custom kernels.
- Data isolation. Models and datasets sit on local disks that only your server uses. A serious provider erases them between customers.
- A fixed cost. A monthly price turns GPU spend into a known line in the budget, with no meter running while a job is paused.
Bare metal or virtual machine?
A dedicated server can be delivered as bare metal or as a single-tenant virtual machine with the GPUs passed through. For GPU compute the performance gap is small, because the GPUs do the work either way. What differs is control: on bare metal you manage the kernel, the drivers and the firmware-level settings yourself; on a virtual machine the provider controls the hypervisor and sometimes the driver version. If you rely on specific kernel modules, GPU topology tools or low-level profiling, ask which one you get.
What to check before you rent one
- GPU model and memory. HBM GPUs (H100, H200, B200, B300 and AMD Instinct) have 3.35 to 8 TB/s of memory bandwidth; GDDR cards (L40S, RTX PRO 6000) have 0.86 to 1.6 TB/s and suit inference and graphics better than large training jobs. Memory decides the largest model you can load: see how much GPU memory you need.
- Interconnect. SXM and OAM GPUs on an 8-GPU board are linked by NVLink or Infinity Fabric at 0.9 to 1.8 TB/s per GPU. PCIe cards exchange data over PCIe: fine for one model per GPU, slower for multi-GPU training.
- Host CPU, RAM and local NVMe. Data loading, tokenisation and checkpoints run on the host. Ask for the CPU cores, the system memory and the NVMe capacity per GPU.
- Network. Check the bandwidth, whether traffic is included and whether you get a public IPv4 address for SSH and serving.
- Images and drivers. A ready image with CUDA or ROCm, PyTorch and vLLM saves the first hour on every new server.
- Billing and commitment. Look for the price per GPU per month, what happens at renewal and whether you can stop at the end of any month.
- Support and security. Who answers when a GPU fails, how fast hardware is replaced, and how disks are wiped between customers.
Before a long run, validate the server you receive: the 8-GPU server checklist covers GPU health, topology, collective bandwidth and a short burn-in.
How much does a dedicated GPU server cost?
Dedicated GPU servers are usually priced per GPU per month. CoreGPU prices, next to the market median (the on-demand list price of specialist GPU clouds over 730 hours):
| GPU | Memory | Per GPU / month | 8-GPU server / month | Market median / GPU |
|---|---|---|---|---|
| NVIDIA B300 | 288 GB HBM3e | $4,899 | $39,192 | $6,552 |
| NVIDIA B200 | 180 GB HBM3e | $4,449 | $35,592 | $5,979 |
| AMD MI355X | 288 GB HBM3E | $2,449 | $19,592 | $3,285 |
| AMD MI350X | 288 GB HBM3E | $2,249 | $17,992 | $3,022 |
| NVIDIA H200 | 141 GB HBM3e | $2,699 | $21,592 | $3,606 |
| AMD MI325X | 256 GB HBM3E | $2,299 | $18,392 | $3,071 |
| AMD MI300X | 192 GB HBM3 | $1,749 | $13,992 | $2,351 |
| NVIDIA H100 | 80 GB HBM3 | $1,999 | $15,992 | $2,716 |
| NVIDIA RTX PRO 6000 | 96 GB GDDR7 | $1,199 | $9,592 | $1,599 |
| NVIDIA L40S | 48 GB GDDR6 | $849 | $6,792 | $1,146 |
USD, excluding taxes. Prices checked Nov 4, 2026; live prices and stock on the pricing page.
Monthly billing beats hourly once you would keep the GPU more than about 540 hours a month, roughly 18 hours a day. The full calculation is in hourly vs monthly GPU rental.
When a dedicated GPU server is the right choice
- Training or fine-tuning runs that last days or weeks.
- Inference that must stay online around the clock, with stable latency.
- Jobs that need the full NVLink or Infinity Fabric bandwidth of an 8-GPU node.
- Sensitive models or data that should not sit on shared hosts.
- Budgets fixed in advance, such as a research grant or a startup runway.
When it is not
For a few hours of testing, a one-off benchmark or a notebook on a small model, an hourly instance or a fractional GPU costs less. A dedicated server earns its price when the GPUs are busy most of the month.
Dedicated GPU servers at CoreGPU
CoreGPU rents dedicated NVIDIA and AMD GPU servers with 1 to 8 GPUs by the month, at least 25% below the market median, with root SSH access, local NVMe and ready CUDA or ROCm images. You pay from a prepaid balance in BTC, ETH, XMR, USDT (TRC-20) or SOL. See every GPU with its live stock, or compare two GPUs side by side.
