Skip to main content

On-premises

This section covers deploying Live Transcoder on on-premises hardware.

HW requirements​

  1. x64 machine with an Nvidia GPU of Turing, Ampere, Ada Lovelace or Blackwell architecture.
  2. disk with 256GB+ space needed for possible HLS output

Prepare host OS - Ubuntu​

Install Ubuntu​

Ubuntu Server 24.04 LTS is supported.

Disable nouveau driver​

nouveau, the open-source driver for NVIDIA GPUs is usually pre-loaded. It must be disabled to install NVIDIA's own driver.

sudo modprobe -r nouveau

Prevent loading of the nouveau kernel module at boot.

echo "blacklist nouveau" | sudo tee /etc/modprobe.d/blacklist.conf

Upgrade OS​

sudo apt update && sudo apt upgrade -y

Install NVIDIA GPU driver​

This installs NVIDIA headless driver (without GUI support).

sudo apt-get update
sudo apt install nvidia-headless-535 nvidia-utils-535 libnvidia-encode-535
sudo modprobe nvidia

Check if NVIDIA driver works​

nvidia-smi

Install Docker CE​

curl https://get.docker.com | sh && sudo systemctl --now enable docker
sudo groupadd docker
sudo usermod -aG docker $USER
newgrp docker
groups

Install NVIDIA Container Toolkit​

curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
&& curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

sudo apt-get update

export NVIDIA_CONTAINER_TOOLKIT_VERSION=1.17.7-1
sudo apt-get install -y \
nvidia-container-toolkit=${NVIDIA_CONTAINER_TOOLKIT_VERSION} \
nvidia-container-toolkit-base=${NVIDIA_CONTAINER_TOOLKIT_VERSION} \
libnvidia-container-tools=${NVIDIA_CONTAINER_TOOLKIT_VERSION} \
libnvidia-container1=${NVIDIA_CONTAINER_TOOLKIT_VERSION}

sudo nvidia-ctk runtime configure --runtime=docker

sudo systemctl restart docker

Install docker-compose​

Docker is installed, but docker-compose is also needed. Let's install it by following these steps.

sudo curl -L https://github.com/docker/compose/releases/latest/download/docker-compose-$(uname -s)-$(uname -m) -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose
docker-compose version

Adjust firewall rules​

Live Transcoder needs some firewall ports opened to work properly. Either open them by:

sudo ufw allow 22/tcp comment 'accept SSH'
sudo ufw allow 80/tcp comment 'accept HTTP & HLS'
sudo ufw allow 443/tcp comment 'accept HTTPS & HLS'
sudo ufw allow 5353:7999/tcp comment 'accept NDI TCP'
sudo ufw allow 5353:7999/udp comment 'accept NDI UDP'
sudo ufw allow 1024:65535/udp comment 'accept MPEG-TS UDP/RTP/SRT'
sudo ufw allow 1935/tcp comment 'accept RTMP inputs'
sudo ufw allow from 192.168.0.0/24 to 224.0.0.0/24 proto igmp comment 'accept IGMP'
sudo ufw --force enable
sudo ufw status

or disable the firewall completely by these. Disabling the firewall completely can improve performance in high-bitrate use cases.

sudo ufw disable
sudo ufw status

Disable mDNS on the host​

The mDNS service mustn't be active in the host OS for NDI discovery to work. To remove it completely from the host OS:

sudo apt-get remove avahi-daemon

Increase max UDP kernel buffer size​

To receive and send streams with overall stream bitrate of 21mbps+, Live Transcoder needs to increase the UDP kernel buffer size. However, the default maximum size is too low so it needs to be increased.

SYSCTL_CONF="/etc/sysctl.d/10-udp-buffer-size.conf"
sudo sh -c "echo '#kernel send a receive windows buffer sizes' > $SYSCTL_CONF"
sudo sh -c "echo 'net.core.rmem_max=262144000' >> $SYSCTL_CONF"
sudo sh -c "echo 'net.core.wmem_max=262144000' >> $SYSCTL_CONF"
sudo sh -c "echo 'net.core.rmem_default=262144000' >> $SYSCTL_CONF"
sudo sh -c "echo 'net.core.wmem_default=262144000' >> $SYSCTL_CONF"
sudo sh -c "echo '' >> $SYSCTL_CONF"

Now load the updated config file.

sudo sysctl -p $SYSCTL_CONF

Increase txqueuelen for network interfaces​

To stream out high-bitrate codecs (e.g., JPEG2000 or JPEG-XS), also txqueuelen parameter should be increased for the respective network interface(s). Below, we set the parameter for all available Ethernet network interfaces (by using KERNEL=="e*"):

sudo bash -c 'cat > /etc/udev/rules.d/80-txqueuelen.rules' << EOF
SUBSYSTEM=="net", ACTION=="add|change", KERNEL=="e*", ATTR{tx_queue_len}="10000"
EOF

To apply the changes immediately, run:

sudo udevadm control --reload-rules && sudo udevadm trigger

Install systemd-coredump​

It's needed for the creation of diagnostic packages when something goes wrong.

sudo apt install systemd-coredump

Prepare host OS - Others​

General OS requirements​

  1. The host instance needs access to the internet and a working DNS
  2. The host OS needs to expose the cgroups v1. New OSs often expose only cgroups v2.
    • You can check currently exposed cgroup versions by: mount | grep '^cgroup' | awk '{print $1}' | uniq
    • cgroup means v1 is exposed, cgroup2 means v2 is exposed (can be both)
    • Kernel can be forced to expose also v1 cgroups by adding kernel parameter systemd.unified_cgroup_hierarchy=0
  3. Docker-CE version 20.0 or later
  4. nvidia-container-toolkit version 1.17.7, there is currently a known issue with 1.17.8-1.18.0. Recommend these versions are avoided or see the following:
  5. Nvidia driver version 535 or later
  6. docker-compose command version 2.5 or later
  7. The host must have these firewall ports/protocols allowed
  • SSH (TCP port 22) - for SSH
  • HTTP (TCP port 80) - needed for access to web UI and HLS output
  • HTTPS (TCP port 443) - needed for secure access to web UI and HLS output
  • MPEG-TS over RTP/UDP/SRT (UDP ports) - add custom ports for your MPEG-TS streams
  • NDI (TCP&UDP) - needed for NDI to work, the easiest way is to open port range 5353-7999 for both TCP & UDP [NDI docs]
  • RTMP input (TCP port 1935) - needed for RTMP server to listen on this port
  • IGMP - for multicast
  1. mDNS disabled - for NDI discovery working in the Docker, typically done by disabling/removing the avahi-daemon
  2. UDP kernel buffers max size for write and read set to 256MB