Proxmox Nvidia Gpu on LXC (pve8-deb12)
install nvidia drivers on proxmox 8 (debian 12) #
proxmox 7 is based on debian 12 so we can follow the official steps:
https://wiki.debian.org/NvidiaGraphicsDrivers#Debian_12_.22Bookworm.22$0
the one thing we must remember is to install PVE kernel headers: pve-headers
update pve #
check you already have the correct pve repos,
if needed you cna add it with this (useful also for the lxc container):
add repo key:
wget https://enterprise.proxmox.com/debian/proxmox-release-bookworm.gpg -O /etc/apt/trusted.gpg.d/proxmox-release-bookworm.gpg
add repo:
nano /etc/apt/sources.list.d/pve-install-repo.list
deb http://download.proxmox.com/debian/pve bookworm pve-no-subscription
now we can update everything to the latest kernel:
apt update
apt upgrade
reboot
add repos for non-free nvidia driver #
since nvidia drivers are in the non-free repos let's add them to our repo list:
edit
nano /etc/apt/sources.list
and add "non-free non-free-firmware" where needed:
deb http://deb.debian.org/debian/ bookworm main contrib non-free non-free-firmware
deb-src http://deb.debian.org/debian/ bookworm main contrib non-free non-free-firmware
deb http://security.debian.org/debian-security bookworm-security main
deb-src http://security.debian.org/debian-security bookworm-security main
deb http://deb.debian.org/debian/ bookworm-updates main
deb-src http://deb.debian.org/debian/ bookworm-updates main
then update:
apt update
install headers and nvidia-driver #
now we can install the pve-headers
apt install pve-headers
and install the nvidia driver:
apt install nvidia-driver
at this point we must reboot:
reboot
debug #
if nvidia-smi won't start might be to errors in the driver installation,
if you have any doubt remove everything from nvidia, fix the headers and reinstall
apt remove nvidia*
check driver installation #
run nvidia-smi to check if everything is correctly installed:
nvidia-smi
output:
13:12:42_root@machine:[~]:#nvidia-smi
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.183.01 Driver Version: 535.183.01 CUDA Version: 12.2 |
|-----------------------------------------+----------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+======================+======================|
| 0 Quadro P400 On | 00000000:26:00.0 Off | N/A |
| 34% 25C P8 N/A / N/A | 1MiB / 2048MiB | 0% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+
+---------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=======================================================================================|
| No running processes found |
+---------------------------------------------------------------------------------------+
container creation #
in proxmox
select your storage for LXC templates and download the debian 12 template
create a container:
for plex you might want to add the folder on a separate mount:
add disk on path: /var/lib/plexmediaserver
30gb
add gpu to container #
on your pve host check that you see someting like: /dev/dri/renderD128
and
grep render /etc/group
render:x:107:
in proxmox gui edit the container, in "Resources" add: "Device Passtrough"
in "Device Path": /dev/dri/renderD128
select advanced
in "GID in CT": "107" (see the grep render before)
fix container permissions for the gpu: #
in /etc/pve/lxc/
add:
# Allow cgroup access
lxc.cgroup2.devices.allow: c 195:* rwm
lxc.cgroup2.devices.allow: c 243:* rwm
# Pass through device files
lxc.mount.entry: /dev/nvidia0 dev/nvidia0 none bind,optional,create=file
lxc.mount.entry: /dev/nvidiactl dev/nvidiactl none bind,optional,create=file
lxc.mount.entry: /dev/nvidia-uvm dev/nvidia-uvm none bind,optional,create=file
lxc.mount.entry: /dev/nvidia-modeset dev/nvidia-modeset none bind,optional,create=file
lxc.mount.entry: /dev/nvidia-uvm-tools dev/nvidia-uvm-tools none bind,optional,create=file
and start your container
now from your container you should see:
$ ls -l /dev/nvidia*
crw-rw-rw- 1 root root 195, 254 Dec 22 20:51 /dev/nvidia-modeset
crw-rw-rw- 1 root root 243, 0 Dec 22 20:51 /dev/nvidia-uvm
crw-rw-rw- 1 root root 243, 1 Dec 22 20:51 /dev/nvidia-uvm-tools
crw-rw-rw- 1 root root 195, 0 Dec 22 20:51 /dev/nvidia0
crw-rw-rw- 1 root root 195, 255 Dec 22 20:51 /dev/nvidiactl
source: https://theorangeone.net/posts/lxc-nvidia-gpu-passthrough/#ref-2-configure-container$0
install nvidia drivers on the container #
nano /etc/apt/sources.list
deb http://deb.debian.org/debian/ bookworm main contrib non-free non-free-firmware
deb-src http://deb.debian.org/debian/ bookworm main contrib non-free non-free-firmware
deb http://security.debian.org/debian-security bookworm-security main
deb-src http://security.debian.org/debian-security bookworm-security main
deb http://deb.debian.org/debian/ bookworm-updates main
deb-src http://deb.debian.org/debian/ bookworm-updates main
wget https://enterprise.proxmox.com/debian/proxmox-release-bookworm.gpg -O /etc/apt/trusted.gpg.d/proxmox-release-bookworm.gpg
nano /etc/apt/sources.list.d/pve-install-repo.list
deb http://download.proxmox.com/debian/pve bookworm pve-no-subscription
apt update
apt upgrade -y
reboot
apt install pve-headers
apt install nvidia-driver
reboot
nvidia-smi
apt install lshw
lshw -c video
root@plex-gpu:~# lshw -c video
*-display
description: VGA compatible controller
product: GP107GL [Quadro P400]
vendor: NVIDIA Corporation
physical id: 0
bus info: pci@0000:26:00.0
logical name: fb0
version: a1
width: 64 bits
clock: 33MHz
capabilities: pm msi pciexpress vga_controller bus_master cap_list rom fb
configuration: depth=32 driver=nvidia latency=0 resolution=800,600
resources: irq:73 memory:fb000000-fbffffff memory:d0000000-dfffffff memory:e0000000-e1ffffff ioport:f000(size=128) memory:c0000-dffff
tips and tricks: #
mount bind on containers: #
in /etc/pve/lxc/
mp99 /host/folder/source,mp=/container/target
mp99 /host/folder/source,mp=/container/target_readonly,ro=1
sources: #
- Previous: Restoring data from a broken disk