Deploy a dedicated GPU server

vL vLLMLLM inference High-throughput LLM serving with an OpenAI-compatible API, split across every GPU of the server. OpenAI-compatible API :8000 PyTorchTraining & fine-tuning PyTorch with Transformers and Accelerate on the OS, ready for multi-GPU training and fine-tuning. OllamaLLM inference Run open models such as gpt-oss, Llama, Qwen or DeepSeek with one command and a simple REST API. Ollama API :11434 JupyterLabNotebooks & IDE JupyterLab on top of the PyTorch template, as a service that starts with the server. JupyterLab :8888 CU ComfyUIImage, video & 3D Node-based image and video generation for Stable Diffusion, Flux, Wan and other models. Web interface :8188 SG SGLangLLM inference Fast LLM and multimodal serving with prefix caching (RadixAttention) and an OpenAI-compatible API. OpenAI-compatible API :30000 OW Open WebUILLM inference A private ChatGPT-style interface for your own models, served by Ollama on the same server. Web interface :3000 NVIDIA PyTorch (NGC)Training & fine-tuning NVIDIA’s monthly PyTorch container, tuned for Hopper and Blackwell, with Transformer Engine. Ax AxolotlTraining & fine-tuning Config-driven fine-tuning (LoRA, QLoRA, full) for Llama, Qwen, Mistral, Gemma and more. Us UnslothTraining & fine-tuning Faster LoRA and QLoRA fine-tuning with less GPU memory, in JupyterLab. JupyterLab :8888 code-serverNotebooks & IDE VS Code in your browser, running on the server, on top of the PyTorch template. VS Code in the browser :8080 LF LLaMA-FactoryTraining & fine-tuning Fine-tune more than a hundred open models from a web interface or the command line. Web interface :7860 NVIDIA Triton Inference ServerLLM inference Serve PyTorch, ONNX, TensorRT and TensorRT-LLM models over HTTP and gRPC, with dynamic batching. HTTP API :8000 NVIDIA NeMoTraining & fine-tuning NVIDIA’s framework for pre-training and fine-tuning large models at scale, built on Megatron Core. JX JAXTraining & fine-tuning JAX with CUDA, plus Flax and Optax, in a Python environment on the OS. TensorFlowTraining & fine-tuning TensorFlow and Keras 3 with GPU support, in a Python environment on the OS. BlenderImage, video & 3D Headless rendering with Cycles on CUDA or OptiX; fastest on GPUs with RT cores (RTX PRO 6000, L40S). CUDA 13Base images Ubuntu with the NVIDIA driver, the CUDA toolkit, cuDNN, NCCL and Docker with GPU access. Ubuntu 24.04 LTSBase images A clean OS with the GPU driver only: install everything yourself, you have root. Custom Docker imageBase images Any public Docker image, started with access to every GPU when the server boots.