Home / Docs / Infrastructure / proxmox

Proxmox VE: GPU passthrough & VM layout

Proxmox VE is used as the hypervisor to isolate workloads and allow clean GPU passthrough.

Host overview

Proxmox VE host reserves 4–6 GB RAM for itself. Guest layout:

flowchart TB
    Host[Proxmox VE Host]
    Host --> VM1[VM: ai-training]
    Host --> VM2[VM: ocr-processing]
    Host --> CT[CT: file-server]

    VM1 --> |Ubuntu Server 24.04| Spec1[vCPUs: 24, RAM: 96 GB, GPU passthrough, 500 GB NVMe]
    VM2 --> |Ubuntu Server 24.04| Spec2[vCPUs: 32, RAM: 24 GB, CPU-only]
    CT --> |Alpine Linux| Spec3[RAM: 2 GB, NFS dataset share]

ASCII summary

Proxmox VE Host (4–6 GB RAM reserved)
├── VM: ai-training (Ubuntu Server 24.04)
│   ├── vCPUs: 24 (leave 8 for host/OCR)
│   ├── RAM: 96 GB (ballooning enabled)
│   ├── GPU: RTX 3070 (PCI passthrough)
│   └── Storage: 500 GB NVMe (datasets + checkpoints)
│
├── VM: ocr-processing (Ubuntu Server 24.04)
│   ├── vCPUs: 32 (CPU-only)
│   ├── RAM: 24 GB
│   └── Purpose: Parallel OCR and text ingestion
│
└── CT: file-server (Alpine Linux)
    ├── RAM: 2 GB
    └── Purpose: NFS dataset share between VMs

BIOS configuration

Enable IOMMU on Proxmox host

  1. Edit GRUB:

    nano /etc/default/grub

  2. Set kernel command line:

    GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=on iommu=pt"

  3. Apply and reboot:

    update-grub
reboot

  4. Verify:

    dmesg | grep -e IOMMU -e AMD-Vi

GPU passthrough (VFIO)

1. Identify GPU PCI IDs

lspci -nn | grep -i nvidia

Example output:

01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GA104 [RTX 3070] [10de:2484]
01:00.1 Audio device [0403]: NVIDIA Corporation GA104 HDMI Audio [10de:228b]

2. Bind GPU to VFIO

echo "options vfio-pci ids=10de:2484,10de:228b disable_vga=1" > /etc/modprobe.d/vfio.conf

3. Blacklist NVIDIA drivers on host

echo -e "blacklist nouveau\nblacklist nvidia" > /etc/modprobe.d/blacklist-nvidia.conf

4. Update initramfs and reboot

update-initramfs -u
reboot

5. Confirm binding

lspci -nnk -d 10de: