Version: v4 (current)

Debian Machine Setup

Steps for manual configuration and provisioning of Debian 12 server systems. These steps will also upgrade a Debain 11 system to Debian 12. This guide assumes and recommends that the user is starting from a fresh installation. If you unfamiliar with the installation process for Debian, see the links below before progressing.

Base Packages

Add required the apt sources and upgrade your system to the latest version.

cat << EOF > /etc/apt/sources.list
deb http://deb.debian.org/debian bookworm main contrib non-free non-free-firmware
deb-src http://deb.debian.org/debian bookworm main contrib non-free non-free-firmware

deb http://deb.debian.org/debian-security/ bookworm-security main contrib non-free
deb-src http://deb.debian.org/debian-security/ bookworm-security main contrib non-free

deb http://deb.debian.org/debian bookworm-updates main contrib non-free
deb-src http://deb.debian.org/debian bookworm-updates main contrib non-free
EOF

Apply system updates and upgrades
Update the package list and upgrade system components prior to installing other software. Reboot after the process completes.
```
# run as root
apt-get update && \
apt-get upgrade -y && \
apt-get full-upgrade -y

reboot
```

Install base utilities

The following is a curated set of base packages that are dependencies for steps later in the guide and helpful in general.

# Run as root
apt-get update && \
  apt-get install -y wireguard \
  ssh-import-id \
  sudo \
  curl \
  tmux \
  netplan.io \
  apt-transport-https \
  ca-certificates \
  software-properties-common \
  htop \
  git-extras \
  rsyslog \
  fail2ban \
  vim \
  gpg \
  open-iscsi \
  nfs-common \
  ncdu \
  zip \
  unzip \
  iotop && \
  sudo wget https://github.com/mikefarah/yq/releases/latest/download/yq_linux_amd64 -O /usr/bin/yq && \
  sudo chmod +x /usr/bin/yq && \
  sudo systemctl enable fail2ban && \
  sudo systemctl start fail2ban

Setup the admin user

Create the user

export NEW_USER="<some new user here>"
useradd -s /bin/bash -d /home/$NEW_USER/ -m -G sudo $NEW_USER

Grant passwordless sudo permission

echo "$NEW_USER ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers

Import an ssh key
If you have a GitHub, GitLab, or Launchpad account, you can use ssh-import-id to install you ssh public-key into your host.
If you do not have one of the above, you can always upload your key manually as described in SSH Copy ID for Copying SSH Keys to Servers from ssh.com.
Example usage for ssh-import-id:
```
# GitHub
sudo -u $NEW_USER ssh-import-id-gh <github username>

# GitLab
URL="https://gitlab.exampledomain.com/%s.keys" sudo -u $NEW_USER ssh-import-id <gitlab username>
```
Add the user to relevant groups
```
usermod -a -G kvm $NEW_USER
```
Create a password for the user
```
passwd $NEW_USER
```

Install Docker

Download the docker GPG key

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | \
sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg

Add the apt package source

echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/debian $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

Update the apt package list and install Docker

sudo apt-get update && sudo apt-get install -y docker-ce

Add the user to the docker group
```
usermod -a -G docker $NEW_USER
```

install Docker Compose (Optional)

The default apt package lists provide a very outdated version of docker compose. Below are the steps required to install a current version.

Find the latest version by visiting https://github.com/docker/compose/releases
```
export COMPOSE_VERSION="2.17.3"
```
Create a directory for the binary
```
mkdir -p ~/.docker/cli-plugins/
```

Download the binary

curl -SL https://github.com/docker/compose/releases/download/v$COMPOSE_VERSION/docker-compose-linux-x86_64 -o ~/.docker/cli-plugins/docker-compose

Make it executable

chmod +x ~/.docker/cli-plugins/docker-compose

Install NVIDIA GPU Drivers (Optional)

Debian's built-in driver installation process is more reliable than Ubuntu's, but readers are still advised to get their driver installaer directly from NVIDIA. Instructions for apt installation are included for completeness.

Download and Install from Nvidia

Install required packages

# Run as root
apt-get install -y gcc \
firmware-misc-nonfree \
linux-headers-amd64 \
linux-headers-`uname -r`

Find your driver version and download the installer
You can find both gaming and data-center drivers using the Nvidia web tool, however cloud and vps users will need to refer to their provider's documentation for installation instructions as many providers will require you to use GRID drivers instead.
- Nvidia's web tool: https://www.nvidia.com/download/index.aspx.
Alternatively, you can download a specific driver version using curl:
```
export DRIVER_VERSION=""

# GeForce Cards
curl --progress-bar -fL -O "https://us.download.nvidia.com/XFree86/Linux-x86_64/$DRIVER_VERSION/NVIDIA-Linux-x86_64-$DRIVER_VERSION.run"

# Datacenter Cards
curl --progress-bar -fL -O "https://us.download.nvidia.com/tesla/$DRIVER_VERSION/NVIDIA-Linux-x86_64-$DRIVER_VERSION.run"
```
Run the Installer
It is required to run the driver from the system console, it cannot be installed from an X-session. If you used a desktop image instead of a server image you will need to disable your session manager.
```
sudo bash "NVIDIA-Linux-x86_64-*.run"
```

Download and Install from Apt

Will not work for data-center cards or hosts which require GRID drivers.

sudo apt-get install nvidia-driver

Install the Nvidia Container Toolkit (Optional)

The Nvidia Container Toolkit allows contains to access the GPU resources of the underlying host machine. Requires that the GPU drivers are already installed on the host. See the official docs here: https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html

Set your system distribution name to Debian 11 as workaround until Nvidia adds official Debian 12 support
```
distribution=debian11
```

Download the gpg key and add the repo to your apt sources

curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
&& curl -s -L https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.list | \
    sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
    sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

Update apt packages and install the container toolkit

sudo apt-get update && sudo apt-get install -y nvidia-container-toolkit

Set nvidia the default container runtime

sudo nvidia-ctk runtime configure --runtime=docker --set-as-default

Restart the docker service
```
sudo systemctl restart docker
```
Test that it is working
Run one of Nvidia's sample workloads to test system functionality.

Install the Prometheus metrics exporter (Optional)

Download the metrics exporter application

wget -O /opt/node_exporter-1.6.1.linux-amd64.tar.gz https://github.com/prometheus/node_exporter/releases/download/v1.6.1/node_exporter-1.6.1.linux-amd64.tar.gz && \

Extract the archive and copy into place

tar -xvf /opt/node_exporter-1.6.1.linux-amd64.tar.gz -C /opt && \
rm /opt/node_exporter-1.6.1.linux-amd64.tar.gz && \
ln -s node_exporter-1.6.1.linux-amd64 /opt/node_exporter

Download and create the system service

wget https://raw.githubusercontent.com/small-hack/smol-metal/main/node-exporter.service && \
sudo mv node-exporter.service /etc/systemd/system/node-exporter.service && \
systemctl daemon-reload && \
systemctl enable node-exporter && \
systemctl restart node-exporter

Debian Machine Setup

Base Packages​

Setup the admin user​

Install Docker​

install Docker Compose (Optional)​

Install NVIDIA GPU Drivers (Optional)​

Download and Install from Nvidia​

Download and Install from Apt​

Install the Nvidia Container Toolkit (Optional)​

Install the Prometheus metrics exporter (Optional)​