Mastering Docker within an Unprivileged Devuan LXC Container on Proxmox: A Comprehensive Guide #

As enthusiasts and practitioners in the realm of homelab and personal server management, we often find ourselves navigating the intricate landscape of virtualization and containerization. Our goal is to achieve maximum efficiency, flexibility, and resource utilization. The Proxmox Virtual Environment is a powerful platform, enabling us to host a multitude of services. For applications such as Immich, Jellyfin, and a plethora of other personal productivity and media tools, Docker has become the de facto standard for deployment. The allure of Docker lies in its ability to package applications and their dependencies into isolated environments, ensuring consistency and simplifying management.

However, a common aspiration among homelab users is to consolidate these Docker workloads within a Linux Container (LXC) rather than a full Virtual Machine (VM). This preference stems from the inherent resource overhead associated with VMs, which typically allocate dedicated, constant resources. For a dynamic Docker host setup, where resource demands can fluctuate significantly based on the number and type of running containers, LXC presents a more lightweight and agile alternative. The efficiency of LXC, with its shared kernel approach, means less RAM and CPU are consumed per instance compared to a VM, allowing for greater density on a single Proxmox node.

This guide, brought to you by revWhiteShadow, delves into a specific, yet common, challenge: running Docker within an unprivileged Devuan LXC container on a Proxmox server. We understand that Proxmox documentation often advises against running Docker within LXC containers, particularly unprivileged ones, and instead recommends using VMs. This recommendation is often rooted in security and privilege concerns. Furthermore, many users, including ourselves, have a preference for specific Linux distributions. In this instance, we are focused on Devuan Linux, a fork of Debian that eschews the use of systemd. This choice is not arbitrary but reflects a conscious decision to utilize a system that aligns with our operational philosophy.

Our investigation into this particular setup has revealed a persistent roadblock. When attempting to install Docker Engine within an unprivileged Devuan container, specifically Devuan 5 (which is based on Debian 12), the installation process via apt fails. The critical error message we encounter is: /etc/init.d/docker: 69: ulimit: error setting limit (Operation not permitted). This error, when invoking invoke-rc.d: initscript docker, action "start" failed, strongly suggests a kernel-level permission issue, often associated with unprivileged container environments.

Intriguingly, when we replicate the exact container configuration—specifically, an unprivileged LXC container with nesting enabled—but utilize Ubuntu 24.04 LTS as the base image, the Docker installation proceeds flawlessly. We can successfully install Docker CE (Community Edition) and even execute the foundational docker run hello-world command. This differential behavior between Devuan and Ubuntu under identical unprivileged LXC conditions begs several crucial questions that we aim to address comprehensively in this article.

Our primary objectives are to understand:

1. The fundamental reasons behind the successful Docker installation in an unprivileged Ubuntu LXC container versus the failure in an unprivileged Devuan LXC container, given identical configurations.
2. Strategies to configure our Devuan LXC container to permit Docker execution without compromising its security posture or resorting to privilege escalation.
3. Viable methods for installing Docker in a “rootless” mode, thereby enabling its operation within the constraints of an unprivileged Devuan LXC container.

By the end of this extensive exploration, we aim to provide a clear, actionable path for successfully deploying Dockerized applications within your Proxmox homelab using an unprivileged Devuan LXC container, overcoming the hurdles that have previously seemed insurmountable.

Understanding LXC, Unprivileged Containers, and Privilege Escalation #

Before we dive into the specifics of Docker and Devuan, it is imperative to establish a foundational understanding of the technologies at play. Proxmox VE leverages LXC for containerization. Unlike traditional VMs, LXC containers share the host system’s kernel. This kernel sharing is the source of their efficiency, but it also introduces certain limitations and security considerations.

An unprivileged LXC container is a container that runs with reduced privileges. In essence, the root user inside the container does not map directly to the root user outside on the host system. Instead, a range of User IDs (UIDs) and Group IDs (GIDs) are mapped. For instance, the root user (UID 0) inside the container might be mapped to UID 100000 on the host, and so on. This separation is a critical security mechanism, designed to prevent a compromised container from gaining elevated privileges on the host.

Nesting refers to the ability of a container to run its own virtualized environment, such as Docker or even another LXC instance. For Docker to function correctly within an unprivileged container, especially when it needs to perform operations that are typically root-level, nesting is usually a prerequisite. This allows the container to create its own namespaces and cgroups, which are fundamental to Docker’s operation.

The “Operation not permitted” error encountered with ulimit is a classic indicator that a process is attempting an operation that its current user context is not allowed to perform. In the context of Docker, ulimit settings are often used to manage resource limits for containers, such as the number of open files or processes. When the ulimit command fails with “Operation not permitted” within an unprivileged container, it signifies that the container’s user mapping is preventing it from altering these system-level resource controls.

Why Docker Thrives in Unprivileged Ubuntu LXC (Nesting Enabled) but Fails in Devuan #

The stark contrast in behavior between Ubuntu and Devuan under identical unprivileged LXC configurations with nesting enabled points towards subtle but significant differences in their default configurations, kernel module availability, or how they handle user-space operations that interact with kernel features like namespaces and cgroups.

Ubuntu’s Robust Namespace and Cgroup Support #

Ubuntu, being a widely adopted distribution for server and container workloads, has historically had excellent out-of-the-box support for the kernel features required by Docker. This includes:

• Namespaces: Docker relies heavily on namespaces (PID, network, mount, UTS, IPC, user, Cgroup) to isolate containers. Ubuntu’s kernel and userspace utilities are typically well-configured to allow the creation and management of these namespaces.
• Cgroups (Control Groups): Cgroups are essential for resource management (CPU, memory, I/O) within containers. Docker uses cgroups to limit and account for resource usage. Ubuntu’s default setup generally includes the necessary cgroup controllers and allows their manipulation.
• overlayfs or aufs: Docker uses storage drivers like overlayfs to manage image layers. Ubuntu systems are typically pre-configured with the necessary kernel modules for these storage drivers to function.
• uidmap and gidmap Configuration: Unprivileged containers rely on user namespace remapping, managed by uidmap and gidmap. Ubuntu’s default user-space tools and configurations are often more aligned with enabling these mappings seamlessly for applications like Docker, even in an unprivileged context. The newuidmap and newgidmap binaries are crucial here.

Devuan’s Systemd-Free Philosophy and Potential Gaps #

Devuan’s core philosophy is to provide a system without systemd. While this is achieved through various means, including alternative init systems like SysVinit or OpenRC, it can sometimes lead to differences in how certain services or system components are managed and configured.

• Init System Differences: Docker’s service management often relies on systemd’s unit files. When running on a non-systemd system, Docker’s startup scripts (like those in /etc/init.d/) must correctly interact with the chosen init system. If there are subtle incompatibilities or if the ulimit settings are not correctly applied by the init script in conjunction with the unprivileged container environment, this can lead to the observed failure.
• User Namespace Utilities and Configuration: The critical factor in unprivileged container operations is the correct setup of user namespace mapping. While Devuan uses uidmap and gidmap similar to Debian, the exact configuration of the subuid and subgid files, and how the newuidmap/newgidmap binaries are invoked or configured, might differ. The “Operation not permitted” error on ulimit strongly suggests that the user namespace mapping is not allowing the Docker daemon to set appropriate limits, even when it’s trying to do so within its mapped user context.
• Kernel Module Availability and Loading: Although LXC containers share the host kernel, certain userspace tools might expect specific kernel modules to be available or loaded in a particular way. It’s possible, though less likely if the base OS is Debian 12-like, that some subtle difference in how Devuan manages kernel modules or their interfaces might play a role, particularly concerning the prctl system calls used by ulimit in certain contexts.
• Default Package Versions and Dependencies: While both Devuan 5 and Debian 12 are based on similar Debian Sid snapshots, the exact versions of docker-ce, related containerd components, runC, and the user-mapping utilities might differ. A specific version of docker-ce might have a bug or a dependency that is not met or behaves differently in the Devuan environment compared to Ubuntu.

The success in Ubuntu suggests that its userland tools and default configurations are more adept at handling the complexities of ulimit and resource control within the user-namespace constraints imposed by an unprivileged container. Devuan’s init scripts or underlying user-mapping utilities might not be correctly setting up the environment for Docker’s daemon to perform these operations, even within its mapped privileges.

Configuring Devuan LXC for Docker Without Compromising Security #

The core concern is to enable Docker’s functionality within the unprivileged Devuan container without weakening the security boundary between the container and the host. Privilege escalation, in this context, means allowing the container to gain access or control it shouldn’t have.

Leveraging lxc.logit and lxc.cap_drop for Fine-Grained Control #

The Proxmox LXC configuration offers powerful options to control capabilities. While we want Docker to run, we can restrict what it can do.

• lxc.cap_add: This directive allows specific Linux capabilities to be granted to the container. Capabilities are fine-grained permissions that allow processes to perform certain privileged operations without granting full root access. For Docker, certain capabilities are essential.
• lxc.cap_drop: Conversely, this directive explicitly drops capabilities. By default, unprivileged containers already have most capabilities dropped. The goal is to add only what’s strictly necessary.

The challenge with Docker in an unprivileged container is that it often requires capabilities that are usually associated with root. The ulimit error suggests that the container process needs to modify process resource limits, which is a privileged operation.

Exploring privileged = true with Caution (Not Recommended for Unprivileged) #

It is important to note that setting lxc.prlimit = "" (which essentially means no limit on ulimit) and lxc.cgroup.devices.allow: a (allow all devices) are sometimes mentioned. However, the most direct way to bypass the ulimit issue would be to set lxc.conf.options.privileged = true within the container’s configuration. However, this is precisely what we want to avoid in an unprivileged container setup. Setting privileged = true effectively bypasses most of the security isolation of an unprivileged container, making it behave much like a privileged container, which defeats the purpose of running unprivileged. We will not pursue this path as it compromises security.

The Role of lxc.mount.entry for Specific Filesystems #

Docker relies on various filesystem mounts for its operation, including tmpfs, devpts, and potentially sysfs and proc for certain functionalities. Within an unprivileged container, these mounts need to be carefully configured.

• lxc.mount.entry: This directive allows for custom mount points. Ensure that essential virtual filesystems are correctly mounted. For Devuan, ensuring devpts is mounted with appropriate options (gid=4,mode=620) is often crucial for pseudo-terminal operations, which can indirectly affect ulimit.

The key is to understand which specific capabilities are being denied by the “Operation not permitted” error and if these can be selectively added. This often involves trial and error and deep inspection of Docker daemon logs and system calls within the container.

Rootless Docker: The Most Promising Avenue for Unprivileged Devuan #

Given the strict security posture of unprivileged containers, the most elegant and secure solution for running Docker within such an environment is to leverage Rootless Docker. Rootless Docker is a mode of operation where the Docker daemon and its containers run entirely under a non-root user’s UID. This inherently aligns with the principles of unprivileged containers.

Understanding Rootless Docker Architecture #

Rootless Docker utilizes several key technologies:

1. User Namespaces: As mentioned, these are fundamental. The rootless Docker daemon runs in its own user namespace, where its “root” user is mapped to a non-privileged UID on the host.
2. subuid and subgid: These files define the range of UIDs and GIDs that a user is allowed to map into user namespaces. The rootless Docker installer typically requires these to be set up correctly for the user running Docker.
3. rootlesskit: This is the core component that enables rootless operation. It starts the Docker daemon and containers within an isolated environment managed by user namespaces, cgroup namespaces, and network namespaces.
4. Port Forwarding: Since the rootless Docker daemon doesn’t listen on privileged ports (like 80 or 443) directly, port forwarding mechanisms are used to expose container ports to the host or externally.

Installing Rootless Docker on Devuan (Step-by-Step) #

The process for installing rootless Docker on Devuan within an unprivileged LXC container should be similar to installing it on a standard Debian system.

1. Prerequisites and Container Setup #

• Unprivileged LXC Container: Ensure your Devuan LXC container is created as unprivileged.
• Nesting Enabled: Crucially, ensure nesting is enabled in the LXC configuration. This is usually done by adding lxc.include = /usr/share/lxc/config/common.conf to your container’s configuration and potentially lxc.cgroup2.host-path = /sys/fs/cgroup if on newer Proxmox versions.
• Kernel Headers/Modules: While Proxmox shares the host kernel, ensure the Devuan container has necessary userspace tools like uidmap, shadow, and xz-utils installed.

  apt update
  apt install -y uidmap shadow xz-utils
  

• User Account: Create a dedicated non-root user within the Devuan container that will run Docker. Let’s call this user dockeruser.

  adduser dockeruser
  

2. Configure subuid and subgid for dockeruser #

This is arguably the most critical step for rootless operation. The dockerd process needs a range of UIDs and GIDs to map its internal “root” user and container processes.

First, check if subuid and subgid exist and have entries for dockeruser. If not, you’ll need to create them. The range is typically assigned by /etc/subuid and /etc/subgid.

• Check existing entries:

  grep dockeruser /etc/subuid
  grep dockeruser /etc/subgid
  

• If no entries, create them: You need to allocate a range of UIDs and GIDs. A common range is 65536 to 131071 (65536 UIDs). The starting UID for dockeruser should be greater than any existing users’ mapped UIDs. Let’s assume a starting UID of 100000.

  # Add a line to /etc/subuid
  echo "dockeruser:100000:65536" >> /etc/subuid
  
  # Add a line to /etc/subgid
  echo "dockeruser:100000:65536" >> /etc/subgid
  

  Important: These ranges should not overlap with the host’s actual UIDs/GIDs or other containers’ mapped ranges if you have specific host-level considerations. The default Proxmox mapping for unprivileged containers often starts at 100000. It’s best practice to ensure your user’s range starts after Proxmox’s default range if it’s lower, or use a distinct high range.

3. Install Rootless Docker #

The official method involves downloading a convenience script.

# Switch to the dockeruser
sudo su - dockeruser

# Download and run the script
curl -fsSL https://get.docker.com/rootless | sh

This script will:

• Download rootlesskit and the necessary Docker components.
• Configure Docker to run in rootless mode.
• Set up necessary environment variables (e.g., DOCKER_HOST).
• Install docker-compose as well, if available in the script’s logic.

4. Configure Environment Variables #

The get.docker.com/rootless script usually sets up environment variables either in ~/.profile or ~/.bashrc. You might need to log out and log back in or source these files.

# Source the profile to load new variables
source ~/.profile

Key variables include:

• DOCKER_HOST="unix:///run/user/$(id -u)/docker.sock": This tells the Docker client where to find the daemon’s socket.
• PATH="$HOME/.local/bin:$PATH": Ensures the rootless Docker binaries are in the PATH.

5. Test Rootless Docker #

After sourcing the profile or logging back in, you should be able to run Docker commands as dockeruser:

docker version
docker ps
docker run --rm hello-world

If these commands execute successfully, you have achieved the goal of running Docker within your unprivileged Devuan LXC container.

Addressing Potential Issues with Rootless Docker on Devuan #

• Socket Permissions: Ensure the docker.sock file created by rootlesskit has the correct permissions for dockeruser to access it. The script usually handles this.
• Networking: Rootless Docker uses its own network stack, often with slirp4netns. This can sometimes have performance implications or compatibility issues compared to Docker running as root. You might need to configure port forwarding explicitly if ports are not accessible.
• systemd Dependency (Workaround): While Devuan eschews systemd, rootlesskit might still have some internal logic or expectations that are easier to manage with systemd services. However, the installation script is designed to work without it.
• Resource Limits (ulimit revisited): The “Operation not permitted” error related to ulimit is precisely what rootless Docker aims to solve. By running in its own user namespace, dockeruser can manage its own resource limits for its processes without needing special host permissions. If you still encounter ulimit errors within the rootless context, it would indicate a fundamental issue with user namespace setup or a bug in rootlesskit or its dependencies on Devuan.

Conclusion: A Secure and Efficient Homelab Setup #

Successfully running Docker within an unprivileged Devuan LXC container on Proxmox is not only possible but also a highly efficient and secure approach for managing your homelab applications. The key lies in understanding the security boundaries of unprivileged containers and leveraging tools like Rootless Docker that are designed to operate within these constraints.

By meticulously setting up user namespace mappings via subuid and subgid and employing the rootless Docker installation script, we can circumvent the ulimit permission errors that plague traditional Docker installations in such environments. This approach respects the unprivileged nature of the LXC container, preventing privilege escalation and maintaining a robust security posture for your Proxmox host.

For users of revWhiteShadow, this means you can confidently deploy applications like Immich and Jellyfin using Docker, all orchestrated within a lightweight Devuan LXC container. This strategy optimizes resource utilization, allowing your single-node Proxmox server to handle a greater number of services without the overhead of full virtual machines. It’s a testament to the flexibility and power of modern containerization and virtualization technologies when understood and applied correctly. We encourage you to explore this method, experiment with your favorite containerized applications, and enjoy a more streamlined and efficient homelab experience.