DEV Community

Peijin Zhang (张沛锦)
Peijin Zhang (张沛锦)

Posted on

Use a container for astronomy data processing

#astronomy #radio #hpc #computing

Radio astronomy needs a lot of software: CASA, WSClean, DP3, LOFAR tools, custom scripts—and installing all of them cleanly on every machine is painful.

Containers let you ship a complete, working environment that runs the same on your laptop, workstation, and HPC.

Below is a short, practical workflow using Podman and the astronrd/linc image, and then an example of building your own image like peijin/lwa-solar-pipehost.

Install Podman

https://podman.io/docs/installation

MacOS, (need brew https://brew.sh/)

brew install podman
Enter fullscreen mode Exit fullscreen mode

Windows, need WSL (follow instructions here:https://learn.microsoft.com/en-us/windows/wsl/install) then inside WSL is a ubuntu.

Or linux

# Debian/Ubuntu sudo apt-get update sudo apt-get install -y podman # Fedora sudo dnf install -y podman
Enter fullscreen mode Exit fullscreen mode

Pull and run image

The astronrd/linc image (from ASTRON) is a great “batteries included” environment for radio interferometric calibration and imaging.

podman pull docker.io/astronrd/linc
Enter fullscreen mode Exit fullscreen mode

Then try to start an interactive shell inside the container:

podman run --rm -it \ -v /your/data/path:/data \ docker.io/astronrd/linc \ /bin/bash
Enter fullscreen mode Exit fullscreen mode

Explanation:

--rm – delete the container when you exit

-it – interactive terminal

-v /your/data/path:/data – mount your local data directory into /data inside the container

docker.io/astronrd/linc – the image to run

/bin/bash – start a bash shell inside

Now you’re inside a fully configured environment with tools like DP3 and WSClean available, without installing them on your host system.

Try WSClean and DP3 inside astronrd/linc

Once inside the container shell, you can test the tools:

wsclean --version DP3 --help
Enter fullscreen mode Exit fullscreen mode

Then exit to escape the shell.

(optional) Build your own Docker/Podman image

Although there are so many plug-and-play-ready prebuilt container images, sometimes when a more customized package is needed, it's more convenient to create an image.

Features:

  • Pin exact versions (WSClean v3.x, DP3 commit XYZ, CASA version, etc.)

  • Add your own pipelines/scripts

  • Pre-configure environment variables, paths, and helper tools

The following is a minimal example of how you might build an image like peijin/lwa-solar-pipehost for an OVRO-LWA–style pipeline host.

Step 1: create Dockerfile

Create a file called Dockerfile:

# Start from a base astronomy image (or just Ubuntu) FROM docker.io/astronrd/linc:latest # or: FROM ubuntu:22.04 # Set a working directory WORKDIR /opt/pipeline # Install extra system packages (example) RUN apt-get update && apt-get install -y --no-install-recommends \ git \ python3 \ python3-pip \ && rm -rf /var/lib/apt/lists/* # Copy your pipeline scripts into the image COPY pipeline/ /opt/pipeline/ # Install Python dependencies RUN pip3 install --no-cache-dir -r requirements.txt # Set environment variables (example) ENV PYTHONUNBUFFERED=1 \ OMP_NUM_THREADS=8 # Default entrypoint (can be a shell, or a pipeline script) ENTRYPOINT ["/bin/bash"]
Enter fullscreen mode Exit fullscreen mode

Of course, you can do other optimizations to include more packages.

Step 2: Build the image

From the directory with your Dockerfile:

podman build -t peijin/lwa-solar-pipehost:latest .
Enter fullscreen mode Exit fullscreen mode

Step 3: Run container with the image 

podman run --rm -it \ -v /your/data/path:/data \ peijin/lwa-solar-pipehost:latest \ /bin/bash
Enter fullscreen mode Exit fullscreen mode

Inside, your pipeline code is already in /opt/pipeline.

Top comments (0)