Skip to content

Containers in ReD

RES

Containers aren't used. Please talk to the ReD Team of you have any interest on this topic.

Parallel Cluster/Open OnDemand

All software presented in Open OnDemand (OOD) as Interactive Apps is deployed via containers in the filesystem and runtime execution is handled transparently to the user via the software Singularity - a container platform that allows you to create and run containers that package up pieces of software in a way that is portable and reproducible.

Using Containers in SLURM Jobs

For interactive, GUI applications in OOD, no effort is needed: our scripts will set up, load, and execute the containers for you. You can see this by starting a terminal session inside one of the GUI applications (e.g. VSCode, RStudio, etc.). The prompt in that session pane will likely be **Apptainer>**, which is the open source, forked version of Singularity.0

For batch or interactive command-line jobs, note the following:

  • Location: All current containers are located at /shared/software/singularity/current_containers/. Note: Please do not use containers outside of this directory unless instructed to do so by a ReD team member.
  • Naming: Containers are usually named via application and possibly version number and/or release date. We try to ensure that the names here do not change much at all over time (across maintenance sessions), as we need to update them on a regular basis (software vulnerability patching).
  • Isolation: Containers are like sandboxes - isolated from the outside environment. That doesn't work if you wish to access your home or project folders. For this reason, one can tie, or bind mount, outside directories to inside the container. You'll have at least two bind mounts: your home folder and your project folder. Determine if you need any others.
  • Enhancements: Additional capabilities like leveraging GPUs must also be flagged when executing the container software, even if software support for the ehancement is baked in(to the container).

With this in mind, let's run an RStudio batch job (taken from the R/RSstudio doc page):

cd /path/to/my/project_folder 


#
# file R_test_script.sh containing our job 
#
# Note this example uses a bash heredoc: when executed, it creates the SLURM job file 
# and then submits it. See https://en.wikipedia.org/wiki/Here_document 

# Create our slurm job script...
cat << EOF > R_test_script.slurm
#!/bin/bash
#SBATCH --job-name=R_test          # Job name
#SBATCH --output=%x_%j.out         # Output file (%x expands to Job name, %j expands to job ID)
#SBATCH --error=%x_%j.err          # Error file
#SBATCH --ntasks=1                 # Number of tasks (processes)
#SBATCH --nodes=1                  # Run on only 1 node
#SBATCH --cpus-per-task=1          # Number of CPU cores per task
#SBATCH --mem=4G                   # Total memory per node
#SBATCH --time=00:30:00            # Time limit (hh:mm:ss)
#SBATCH --partition=urcdtest-med   # Partition name
#SBATCH --open-mode=append         # Ensure that log files are appended (vs truncate) on job restarts

# Set up 'global' variables
LAB_DIR=/data/labs/harvardj_lab
PROJ_DIR=$LAB_DIR/projects/my_project

# Set up container details
#  Binds are needed to tie outside directories to locations inside. We are using shorthand format
#   since inside and outside are the same (format: outside/[:inside/]). Omit
#   last part if not part of NSAPH FASSE.
#  Omit GPU option if not needed
#
export R_VERSION=4.5.2
export R_LIBS_USER=$HOME/R/R_${R_VERSION}
container_rstudio=/shared/software/singularity/current_containers/urcds-rstudio_${R_VERSION}.sif
binds="-B /shared,/data,/nsaph-fasse"
gpu_support="--nv"

# Now run our code inside the container
# Entry point can be 'exec', 'run', ...
singularity exec \
  $binds $gpu_support \
  ${container_rstudio} \
  Rscript my_code.R

# now report our job efficiency when exiting
seff $SLURM_JOBID

EOF

Note the following:

  • We're pointing to the RStudio singularity image in the fileystem with a shell variable $container_rstudio.
  • We're noting bind mounts via another shell variable $binds using the program option (or flag) -B (bind) and the base folders of our home directory (/shared) and our project folder (/data) separated with a comma ,.
  • We're including GPU support via yet another variable $gpu_support and the flag --nv (presumably referring to NVidia).

These program options (or flags or arguments) – we are optional - we include when we call the singularity program, in the format:

singularity exec [options] container program [programoptions]
There are many other options. Please see the Singularity or Apptainer docs for more details.

Now submit the job from the OOD terminal window, Cluster Access > >_urcds-red-pc-prod Shell Access:

# again, ensure we're at the right location
cd /path/to/my/project_folder 

# now submit the job 
sbatch code/R_test_script.slurm

When your job is dispatched, your SLURM script will run, and the indicated container will be loaded, modified by the given options, and RScript will be executed inside the container, which will subsequently execute your code file.

For more details, please see our SLURM docs on batch jobs or interactive command-line jobs.

Importing Vendor or Custom Containers

Do not bring containers in via Globus unless instructed to do so by ReD team members

At this time we are not resourced to assist users in importing containers made outside ReD: we would need to register, vulnerability scan, and subsequently track the location of all custom containers. Due to security requirements for this environment, we need to remediate all softawre vulnerabilities on a regular basis (hence our twice-monthly scheduled maintenance sessions). Please check back with us later on this.

Building Containers inside ReD

This is not supported at this time. In detail:

Technically this is possible. However, often containers are built on base images, which are pulled from public registries or vendor sites. Since the environment does not permit internet access (for the most part), pulling these base images is not possible.

In theory one could use a base container image that we already have in ReD. Beyond this, one could only add (new) software that is available via our PPM softare mirror.

We hope to provide a more robust, capable service around this in the future.

Updated: Dec 16, 2025