Slurm Docker Integration

Docs / Administrator Guide / Slurm / Slurm Docker Integration
Administrator Guide

Slurm Docker Integration

Configuring Slurm for docker integration with spillbox flow

Make sure that you set these values in your slurm.conf:

LaunchType=launch/slurm
ProctrackType=proctrack/cgroup
PrologFlags=Alloc
JobContainerType=job_container/none

Assuming your spillbox installation is done /spillbox_dir, you can find the slurm-setup.sh in your spillbox installation directory (i.e /spillbox/terraform/bin/slurm-setup.sh)

Make sure that this script can be accessible on all job execution machine , or you can have this script in some shared directory and use that path in the below value

Set this value in the slurm.conf

Prolog=/spillbox/terraform/bin/slurm-setup.sh

In client tar file , you can find wrappers for all slurm commands in the /spillbox_client/slurm directory (Assuming the client installation is done on /spillbox_client) , add them to the $PATH , so that when you submit job , it uses spillbox slurm commands

You can submit a job using :

/spillbox_client/slurm/srun -n 1 ~/dummy.sh
/spillbox_client/slurm/sbatch -o ~/outfile -e ~/errfile ./dummy.sh

To unlock debug capability,

1.you can set SPILLBOX_CONTAINER_DEBUG=1 and submit the job ,so the failed/finished container will not be removed 2.To login into the job conatiner , you can run : /spillbox_client/slurm/sbatch -jobattach <jobid>

Limitation

  1. Ctrl+c will not kill the srun job
  2. srun can't be done inside sbatch jobs
  3. Prolog script for the job is not allowed

Common FAQ's

Q. Can user give their custom prolog script ?

Ans: No, Spillbox will not allow user to use their custom prolog ,if users want to run the jobs on docker container

Q. How to tell spillbox to not remove failed/finished job conatiner?

Ans: Set the environment variable SPILLBOX_CONTAINER_DEBUG=1 before submitting the job

On this page
💬 Ask a Question