LSF Multicluster

Assuming LSF has been configured with Docker and application profile, refer to the spillbox_lsf_docker_app_profile_configuration file, which contains steps to create a docker application profile. Assuming one LSF cluster on the cloud and another one on the on-prem has been created.

If a user wants to run their jobs both on-prem and the cloud , following two scenarios describe the setup for multicluster LSF.

CASE 1

If a user wants to run all the jobs on a docker container in the cloud and run as normal jobs on the on-prem (job runs without docker) You need to configure your LSF with a docker application profile in the lsb.application on the cloud and on the on-prem you need to create normal application profile with the following contents

Begin Application 
NAME = splapp 
JOB_STARTER =/<full_path_to_/exewrapper.sh> %USRCMD
End Application

CASE 2

If a user wants to run all the jobs in a docker both on-prem and on the cloud. You need to configure the LSF with docker application profile in the lsb.application on both on the cloud and on-prem.

Note : The application name should be same in both the cloud and on -prem LSF cluster configuration. Spillbox shipped exewrapper.sh should be accessible on all job executing machines. You can use this application profile name when you are using spillbox bsub wrapper which will be in lsf directory in the client tar file.

Example entry of lsb.application to support the job execution without container

Begin Application 
NAME = splapp 
JOB_STARTER =/<full_path_to_/exewrapper.sh> %USRCMD
End Application

Example entry lsb.application to support the job execution with container

Begin Application 
NAME = splapp 
CONTAINER = docker[image(spillbox) options(--net=host --privileged -v /splbox/terraform/bin:/spillbox -v /cache_dir:/spillbox_workdir -v /usr/local/lsf:/usr/local/lsf -e SPILLBOX_SITE=<hostname> --entrypoint /spillbox/bapp_entrypoint.sh)] 
DESCRIPTION = Docker User Service
EXEC_DRIVER=context[user(root)] starter[/splbox/etc/docker-starter.py] controller[/splbox/etc/docker-control.py] monitor[/splbox/etc/docker-monitor.py] 
End Application