Docker to Apptainer Conversion¶

Most HPC clusters do not permit Docker because the Docker daemon requires root privileges. Apptainer (formerly Singularity) is the standard container runtime on HPC: it runs as an unprivileged user, mounts host filesystems safely, and passes through GPU devices.

This guide shows how to take an existing pyCyto Docker image and use it on an HPC cluster via Apptainer.

Why Apptainer on HPC¶

Concern	Docker	Apptainer
Root required	Yes (daemon)	No
Cluster availability	Rarely	Standard
GPU (`--nv`)	`--gpus all`	`--nv`
User namespace mapping	Complex config	Automatic
Image format	Layered tarball	Single `.sif` file
Bind-mount syntax	`-v host:container`	`--bind host:container`
Run syntax	`docker run --rm -it IMAGE CMD`	`apptainer exec SIF CMD`

Build Workflow¶

        graph LR
    A[Dockerfile] --> B[docker build]
    B --> C[docker push to registry]
    C --> D[apptainer pull to .sif]
    D --> E[apptainer exec on cluster]
    A2[containers/apptainer .def] --> D

    classDef dockerNode  fill:#0369a1,color:#fff,stroke:#075985
    classDef apptainerNode fill:#0d7377,color:#fff,stroke:#0a5c60
    class A,B,C dockerNode
    class A2,D,E apptainerNode

Step 1 — Build the Docker Image Locally¶

# From the repo root
docker build \
    --pull \
    -f containers/docker/cyto-gpu.Dockerfile \
    -t cyto-gpu:latest \
    .

Step 2a — Build SIF from Local Docker Image¶

If Apptainer is available on the same machine where you built the Docker image:

apptainer build containers/images/cyto-gpu.sif \
    docker-daemon:cyto-gpu:latest

Step 2b — Pull SIF from a Registry¶

If you pushed the image to a registry (GitHub Container Registry, Docker Hub, etc.):

# From GitHub Container Registry
apptainer pull containers/images/cyto-gpu.sif \
    docker://ghcr.io/bpi-oxford/cyto-gpu:latest

# From Docker Hub
apptainer pull containers/images/cyto-gpu.sif \
    docker://bpiox/cyto-gpu:latest

Step 2c — Build SIF from an Apptainer Definition File¶

The repo ships definition files under containers/apptainer/:

apptainer build containers/images/cyto-gpu.sif \
    containers/apptainer/cyto-gpu.def

A minimal definition file that wraps the Docker image:

Bootstrap: docker
From: ghcr.io/bpi-oxford/cyto-gpu:latest

%post
    # Any extra layer installs here
    pip install --no-cache-dir pycyto==0.1.0

%environment
    export PYTHONPATH=/workspace:$PYTHONPATH

%runscript
    exec "$@"

Step 3 — Run on HPC with Apptainer¶

# GPU-enabled execution
apptainer exec --nv \
    --bind $(pwd):/workspace \
    --bind /gpfs/data:/data \
    containers/images/cyto-gpu.sif \
    pixi run -e gpu python scripts/benchmark/run_benchmark.py

# Interactive shell for debugging
apptainer shell --nv \
    --bind $(pwd):/workspace \
    containers/images/cyto-gpu.sif

--nv passes through the host NVIDIA driver and all GPUs automatically — no extra flags needed.

Step 4 — SLURM + Apptainer sbatch Pattern¶

#!/bin/bash
#SBATCH --job-name=cyto-cellpose
#SBATCH --partition=gpu_short
#SBATCH --gres=gpu:a100-pcie-40gb:1
#SBATCH --cpus-per-task=8
#SBATCH --mem=32G
#SBATCH --time=04:00:00

module load apptainer   # or: module load singularity

apptainer exec --nv \
    --bind ${REPO_ROOT}:/workspace \
    --bind ${DATA_ROOT}:/data \
    ${REPO_ROOT}/containers/images/cyto-gpu.sif \
    pixi run -e gpu python distributed/segmentation/batch_cellpose.py \
        --pipeline configs/pipelines/my_pipeline.yaml \
        --patch 0

The REPO_ROOT and DATA_ROOT environment variables should be set in your SLURM submission script or exported in your .bashrc.

Configure the SIF Path¶

Set gpu_sif in scripts/benchmark/benchmark.user.toml (gitignored) so the benchmark runner uses the container automatically:

[containers]
gpu_sif = "containers/images/cyto-gpu.sif"

Leave empty ("") to fall back to bare-metal pixi environments.

Directory Layout¶

containers/
├── docker/                      # Dockerfiles (source of truth for image content)
│   ├── cyto-gpu.Dockerfile
│   └── cyto-cpu.Dockerfile
├── apptainer/                   # Apptainer .def files (HPC build recipes)
│   ├── cyto-gpu.def
│   └── build.sh                 # convenience build script
└── images/                      # built .sif files (gitignored — large binaries)
    └── cyto-gpu.sif

.sif files under containers/images/ are excluded from git (listed in .gitignore). Rebuild them from the .def files or pull from the registry.