Choose an image when you reserve your server. You can also start from a clean OS and install everything yourself: you have root access.
Available images
| Image | Contents | GPUs |
|---|---|---|
ubuntu-24.04 | Ubuntu 24.04 LTS, GPU driver only | All |
cuda | Ubuntu 24.04, NVIDIA driver, CUDA 13 toolkit, cuDNN, NCCL, Docker + NVIDIA container runtime | NVIDIA |
pytorch-cuda | The cuda image plus the latest stable PyTorch, Transformers and Accelerate | NVIDIA |
vllm-cuda | pytorch-cuda plus vLLM, ready to serve OpenAI-compatible endpoints | NVIDIA |
rocm | Ubuntu 24.04, ROCm 7, RCCL, Docker with GPU access | AMD |
pytorch-rocm | The rocm image plus PyTorch for ROCm | AMD |
vllm-rocm | pytorch-rocm plus vLLM for ROCm | AMD |
Versions and updates
Images track the latest stable releases at the time your server is prepared. Run nvidia-smi or rocm-smi to see the exact driver version, and pip list for Python packages. You can upgrade anything yourself.
Bring your own image
Need a specific OS or driver? Start from ubuntu-24.04 and install what you need, or ask us to prepare a custom image for a private allocation.
Serving a model with vLLM
vllm serve meta-llama/Llama-3.1-70B-Instruct --tensor-parallel-size 8 --port 8000
curl http://localhost:8000/v1/modelsExpose the port through an SSH tunnel or your own reverse proxy with authentication; servers have no public HTTP endpoint by default.