Tutorials

Secure Your Docker Host: Rootless, Hardened, and Upgradable

Secure Your Docker Host: Rootless, Hardened, and Upgradable

A hardened container host is not simply a normal server with Docker installed. The daemon’s privilege boundary, published ports, credentials, logs, backups, and upgrade path all become part of the production design.

This tutorial builds a deliberately small but complete system: a rootless Docker daemon runs an authenticated Nginx service on port 8080, while nftables limits access to approved networks. The workload has resource limits, rotated logs, health checks, a protected secret, consistent backups, and a tested rollback path.

Prerequisites and operating assumptions

Use a dedicated, systemd-based Debian or Ubuntu host with cgroup v2, a stable IP address, working time synchronization, and administrative access through a console or an existing SSH session. The examples assume the official Docker package repository is already configured.

You will need two real network ranges:

  • An administration network permitted to reach SSH.
  • A service or reverse-proxy network permitted to reach port 8080.

The examples use 10.20.0.0/16 and 10.30.0.0/16. Replace them before applying the firewall. Port 8080 carries plain HTTP, so expose it only to a trusted private network, VPN, or TLS-terminating reverse proxy. Do not publish it directly to an untrusted Internet path.

Keep the current administrative session open while changing firewall rules. A syntax check cannot detect an incorrect source network.

Architecture and trade-offs

Docker runs as the unprivileged account appsvc. Container root is mapped into that account’s subordinate user-ID range rather than host root. A container escape therefore does not immediately become a root compromise of the host.

Rootless mode has costs. Its user-space networking path can consume more CPU than native bridge networking, low-numbered ports require additional design, and resource limits depend on cgroup v2 delegation. These are reasonable trade-offs for an internal web workload, but benchmark high-throughput or latency-sensitive services before standardizing on the pattern.

The host layout is intentionally explicit:

/home/appsvc/edge-site/
├── compose.yaml
├── nginx/
│   └── nginx.conf
└── site/
    └── index.html

/home/appsvc/.config/edge-site/
└── htpasswd

/home/appsvc/backups/
└── edge-site-TIMESTAMP.tar.zst

Install the rootless runtime

Run the following as host root. The dedicated account has no password and should not receive broad sudo access.

apt-get update
apt-get install -y \
  docker-ce-cli docker-ce-rootless-extras docker-compose-plugin \
  uidmap dbus-user-session slirp4netns fuse-overlayfs \
  nftables openssl zstd curl

adduser --disabled-password --gecos "" appsvc
loginctl enable-linger appsvc

install -d -o appsvc -g appsvc -m 0750 \
  /home/appsvc/edge-site \
  /home/appsvc/edge-site/nginx \
  /home/appsvc/edge-site/site

install -d -o appsvc -g appsvc -m 0700 \
  /home/appsvc/.config/edge-site \
  /home/appsvc/backups

Confirm that appsvc has entries in both /etc/subuid and /etc/subgid. Do not invent overlapping ranges manually; use the account-management mechanism appropriate to the distribution if either entry is absent.

Start the lingering user manager, then install Docker as appsvc. These commands are issued by root, but the setup tool and daemon run under the service account:

app_uid="$(id -u appsvc)"
test -n "$app_uid"
test -S "/run/user/$app_uid/bus"

sudo -u appsvc \
  XDG_RUNTIME_DIR="/run/user/$app_uid" \
  dockerd-rootless-setuptool.sh check

sudo -u appsvc \
  XDG_RUNTIME_DIR="/run/user/$app_uid" \
  dockerd-rootless-setuptool.sh install

sudo -u appsvc \
  XDG_RUNTIME_DIR="/run/user/$app_uid" \
  systemctl --user enable --now docker

If the bus socket is not present, start the appsvc user manager through loginctl or create a real login session, then repeat the check. Do not fall back to a root-owned Docker socket merely to bypass a user-session problem.

Build the workload

Create the authentication secret

Run this and all subsequent Docker commands as appsvc. The password is read without terminal echo and is passed to OpenSSL through standard input rather than a command-line argument.

cd /home/appsvc/edge-site
umask 077

IFS= read -r -s -p "Operator password: " site_password
printf '\n'
test -n "$site_password"

password_hash="$(
  printf '%s\n' "$site_password" | openssl passwd -6 -stdin
)"
unset site_password

printf 'operator:%s\n' "$password_hash" \
  > /home/appsvc/.config/edge-site/htpasswd
unset password_hash

chmod 0600 /home/appsvc/.config/edge-site/htpasswd

A Compose secret is not a vault and is not encrypted merely because it appears under /run/secrets. Its value is that the credential stays out of the image, environment, Compose file, and process arguments. Protect its host file and every backup containing it.

Configure Nginx

Create /home/appsvc/edge-site/nginx/nginx.conf with this configuration:

user nginx;
worker_processes auto;
error_log /dev/stderr info;
pid /run/nginx/nginx.pid;

events {
    worker_connections 1024;
}

http {
    include /etc/nginx/mime.types;
    default_type application/octet-stream;

    access_log /dev/stdout combined;
    sendfile on;
    keepalive_timeout 30;
    server_tokens off;

    server {
        listen 8080;
        root /usr/share/nginx/html;

        location = /healthz {
            access_log off;
            default_type text/plain;
            return 200 "ok\n";
        }

        location / {
            auth_basic "Restricted";
            auth_basic_user_file /run/nginx/htpasswd;
            try_files $uri $uri/ =404;
        }
    }
}

Create site/index.html containing a simple paragraph such as <p>Rootless service is healthy.</p>. In a real deployment, this directory can hold generated documentation or another read-only site.

Define the container

Create /home/appsvc/edge-site/compose.yaml:

services:
  site:
    image: docker.io/library/nginx:stable-alpine
    container_name: edge-site
    restart: unless-stopped
    ports:
      - "0.0.0.0:8080:8080"
    volumes:
      - ./nginx/nginx.conf:/etc/nginx/nginx.conf:ro
      - ./site:/usr/share/nginx/html:ro
    secrets:
      - basic_auth
    command:
      - /bin/sh
      - -ec
      - |
        cp /run/secrets/basic_auth /run/nginx/htpasswd
        chmod 0444 /run/nginx/htpasswd
        exec nginx -g 'daemon off;'
    read_only: true
    tmpfs:
      - /run/nginx:size=1m,mode=0755
      - /var/cache/nginx:size=16m,mode=0755
    security_opt:
      - no-new-privileges:true
    cap_drop:
      - ALL
    cap_add:
      - CHOWN
      - SETGID
      - SETUID
    pids_limit: 100
    mem_limit: 256m
    cpus: 1.0
    healthcheck:
      test: ["CMD", "wget", "-qO-", "http://127.0.0.1:8080/healthz"]
      interval: 30s
      timeout: 3s
      retries: 3
      start_period: 10s
    logging:
      driver: local
      options:
        max-size: "10m"
        max-file: "5"

secrets:
  basic_auth:
    file: /home/appsvc/.config/edge-site/htpasswd

The image’s narrowly scoped capabilities exist inside the rootless user namespace. They allow Nginx to initialize directories and drop privileges, but they do not grant corresponding host capabilities. The read-only root filesystem and bounded temporary filesystems reduce both persistence opportunities and uncontrolled disk growth.

Start the service:

cd /home/appsvc/edge-site
docker compose config --quiet
docker compose pull
docker compose up -d
docker compose ps

Enforce the host firewall

Rootless Docker does not provide a substitute for host ingress policy. Create /etc/nftables.d/edge-host.nft as root, after replacing both example networks:

table inet edge_host {
    chain input {
        type filter hook input priority 0;
        policy drop;

        ct state invalid drop
        ct state established,related accept
        iifname "lo" accept

        ip protocol icmp accept
        ip6 nexthdr ipv6-icmp accept

        ip saddr 10.20.0.0/16 tcp dport 22 accept
        ip saddr 10.30.0.0/16 tcp dport 8080 accept
    }

    chain forward {
        type filter hook forward priority 0;
        policy drop;
    }

    chain output {
        type filter hook output priority 0;
        policy accept;
    }
}

Inspect existing nftables, firewalld, or UFW ownership before continuing. Multiple firewall managers should not compete for the same hooks. Validate the fragment before loading it:

nft --check --file /etc/nftables.d/edge-host.nft
nft --file /etc/nftables.d/edge-host.nft
nft list table inet edge_host

For persistence, ensure /etc/nftables.conf contains exactly one include "/etc/nftables.d/*.nft" line, validate the complete file with nft --check --file /etc/nftables.conf, and enable nftables.service. Test a second SSH connection from the approved administration network before closing the original session.

Test security and failure paths

From an allowed service-network machine, verify health, rejection, and successful authentication:

curl --fail --silent --show-error --max-time 2 \
  http://HOST_ADDRESS:8080/healthz

test "$(
  curl --silent --output /dev/null \
    --write-out '%{http_code}' \
    --max-time 2 \
    http://HOST_ADDRESS:8080/
)" = "401"

curl --fail --user operator \
  --max-time 5 \
  http://HOST_ADDRESS:8080/

Also test from a network outside the permitted range; the TCP connection should time out or be dropped. Stop the user daemon and confirm the service becomes unavailable, then restart it with systemctl --user start docker. Finally, reboot the host and verify that lingering starts both the rootless daemon and the restart-managed container.

Logging, observability, and capacity

Use docker compose logs --since 15m site for access and error output, and journalctl --user -u docker --since today for daemon events. The local logging driver rotates container logs, but rotation is not centralized retention. Forward selected host and application logs to a remote collector if incident investigation must survive host loss.

Monitor container health, restart count, memory, process count, filesystem usage, and the rootless daemon’s user service. docker stats --no-stream provides a useful spot check, not durable monitoring.

If memory or CPU limits appear ineffective, verify that the host uses cgroup v2 and that the user manager delegates controllers. Do not silently remove limits: treat unavailable enforcement as a deployment failure or document an explicit capacity-control alternative.

Create and test consistent backups

The example is static, but the backup procedure stops the workload to establish a clear consistency boundary. Database-backed services should normally use their native snapshot or dump mechanism instead of copying live data files.

cd /home/appsvc/edge-site
umask 077

backup_dir=/home/appsvc/backups
stamp="$(date -u +%Y%m%dT%H%M%SZ)"
archive="$backup_dir/edge-site-$stamp.tar.zst"

test -d "$backup_dir"
test ! -e "$archive"

docker compose stop
trap 'docker compose start' EXIT

tar --create --zstd --file="$archive" \
  compose.yaml nginx site \
  -C /home/appsvc .config/edge-site

sha256sum "$archive" > "$archive.sha256"

docker compose start
trap - EXIT

restore_dir="$(mktemp -d /home/appsvc/restore-test.XXXXXX)"
tar --extract --zstd --file="$archive" --directory="$restore_dir"
test -f "$restore_dir/compose.yaml"
test -f "$restore_dir/.config/edge-site/htpasswd"

Inspect the restoration directory and remove that exact directory when finished. Copy encrypted backups to separate storage with independent credentials and retention. A backup remaining only on the Docker host is not protection against host loss.

Upgrade with an automatic rollback

The mutable stable-alpine tag makes controlled updates convenient, while the previous local image ID provides a rollback target. Run upgrades from the project directory:

set -eu

cd /home/appsvc/edge-site
image_ref=docker.io/library/nginx:stable-alpine
old_image="$(docker inspect --format '{{.Image}}' edge-site)"
test -n "$old_image"

docker compose pull site

if ! docker compose up -d --force-recreate site; then
    docker tag "$old_image" "$image_ref"
    docker compose up -d --pull never --force-recreate site
    exit 1
fi

healthy=false
for attempt in $(seq 1 30); do
    if curl --fail --silent --output /dev/null --max-time 2 \
        http://127.0.0.1:8080/healthz; then
        healthy=true
        break
    fi
    sleep 2
done

if test "$healthy" != true; then
    docker compose logs --tail 100 site
    docker tag "$old_image" "$image_ref"
    docker compose up -d --pull never --force-recreate site
    exit 1
fi

docker image inspect "$old_image" > /dev/null
docker compose ps

For a larger fleet, promote tested image digests through environments instead of allowing each host to pull a moving tag independently. Keep the previous image until application tests and an observation window have passed.

Common failures

  • The rootless daemon disappears after logout: confirm loginctl show-user appsvc reports lingering enabled and that the user service is enabled.
  • The container is healthy locally but unreachable remotely: inspect the published address with docker port edge-site, then examine nftables counters and the client’s source address.
  • Nginx repeatedly restarts: inspect its logs. Typical causes are an unreadable secret, malformed configuration, or a missing writable tmpfs path.
  • Limits are ignored or rejected: check cgroup v2 and user-service delegation rather than assuming rootless mode can enforce controllers unavailable to the account.
  • An upgrade fails before health testing: preserve the old image ID and run the documented rollback. Do not prune images as part of the upgrade transaction.

Final verification checklist

  • The Docker daemon runs as appsvc, not root, and survives a reboot.
  • Only approved networks can reach SSH and port 8080.
  • Unauthenticated application requests return HTTP 401.
  • The health endpoint succeeds with a bounded timeout.
  • The container has a read-only root filesystem, minimal capabilities, and resource limits.
  • Logs rotate locally and important events have an external retention plan.
  • A backup checksum exists and a restoration was successfully extracted.
  • The previous image remains available and rollback has been exercised.

Hardening works best when it changes routine operations, not just installation defaults. A rootless daemon limits privilege, but the firewall limits exposure; secrets discipline limits leakage; rotation limits disk pressure; backups limit loss; and rehearsed rollback limits upgrade risk. The durable result is not an invulnerable host. It is a host whose failures are constrained, visible, and recoverable.

Blog author portrait

Mihajlo

I’m Mihajlo — a developer driven by curiosity, discipline, and the constant urge to create something meaningful. I share insights, tutorials, and free services to help others simplify their work and grow in the ever-evolving world of software and AI.

Search

Categories

Recent Posts

Wander Europe's Ancient Aisles for Your Next Creative Awakening
Design Your Days for Deep Work and Lasting Fulfillment
Mastering PHP's Asynchronous Capabilities for Snappy Backend Performance
Beyond Autocomplete: Integrating AI Agents for Real Software Engineering Impact
The Remote Team Playbook for Sustainable Technical Excellence