Ans:   Create  container image aliases in file terraform/etc/spillbox_image_map having entry
        of form:
             <short_name>=<host>:<port>/<image>

       It is expected that the admin will  push these images  on registry servers (<host>:<port>)
       on their own.  Users can select image at runtime:
            splrun run -image <short_name> <cmd> <args>

Ans:   Create a NFS Model mapped config parameters in file terraform/etc/spillbox_nfsModel_config
        having the entries of form :
        model_nfs_size_<machine Model short name>="<cache size class>"
        model_nfs_submitopt_<machine Model short name>="<job manager configuation>"
        model_nfs_queue_<machine Model short name>="<queue name>"

       A model carries three independent settings, so none has to encode
       the others:
         model_nfs_size_<model>      the NFS cache size class: low or moderate/high
                                     (moderate and high currently give identical tunables).
         model_nfs_submitopt_<model> the job manager submit options for the NFS
                                     server job. These are appended to
                                     job_submit_command.
         model_nfs_queue_<model>     the job manager queue/partition the NFS
                                     server job is submitted to. Translated per
                                     job manager: lsf/uge -> "-q <queue>",
                                     slurm -> "-p <queue>".

       With the shipped config, nfs_model resolves to these tunables:
         nfs_model   size class   Dir_Chunk   Cache_Size   NParts   HWMarks
         small       low         1024        32633        29       1000000
         medium      moderate     4096        326023       61       1000000
         large       high         4096        326023       61       1000000

        example:
        model_nfs_size_medium="moderate"     use the moderate NFS cache tunables
        model_nfs_submitopt_medium="-n 4"    allow the LSF job manager to allocate 4
                                             slots of CPU to the NFS server
        model_nfs_queue_medium="gpuqueue"    submit the NFS server job to the
                                             "gpuqueue" queue

       Any of the three may be left out. A model with only model_nfs_submitopt_
       set gets the low cache tunables; a model with only a size class is
       submitted with no extra job manager options; a model with no queue set is
       submitted to the job manager's own default queue.

       To configure the default model, IT team shall name it with the "default"
       parameter, and define that model like any other
       example: default="small"
                model_nfs_size_small="low"
                model_nfs_submitopt_small="-n 2"
                model_nfs_queue_small=""

       However the client shall override the default with other predifined config options given by IT team.
       By providing the "nfs_model" config paramter in project.config file from client end
       example: nfs_model="medium"
       With nfs_model defined , the NFS server shall allocate 4 slots of CPU to NFS server ,
       use the moderate cache tunables, and submit to the "gpuqueue" queue, eventhough the default is small.

       Model names must be made of letters, digits and underscores only.

Ans:   If Checkpoint is not enabled, job will restart from beginning on spot preemption.
    To enable running before the preemption, checkpoint needs to be done at regular intervals,
    controlled by user job by job. To enable checkpoint container should have Spillbox
    version of crui executable. Please add below lines in your container Dockerfile:
         RUN yum -y install asciidoc xmlto libbsd-devel libnet libnet-devel protobuf protobuf-c protobuf-c-devel protobuf-compiler protobuf-devel protobuf-python libnl3-devel libcap-devel python3 nftables
         RUN (cd /tmp/; wget https://s3.us-west-1.amazonaws.com/spillbox.io-public/criu/criu.tar; tar xvf criu.tar; cd /tmp/criu-criu-dev/; make clean ; make  install)

    criu will need NFS space to keep images. Please provide the value for checkpoint_image_dir in config file (see above).

Ans:  Set these values : checkpoint_image_dir, checkpoint_objstore, checkpoint_bucket, checkpoint_objport (terraform/etc/spillbox_config)
      Run the script :: terraform/objstore.sh <aws|google|azure> <create|delete|prune>
            - It will create S3 bucket to store data and will create redis database to store its metadata.

Ans:    - To enable global cache redundancy, ensure that you have two global cache projects configured:
            Project_A – Primary global cache
            Project_B – Secondary global cache
          When the primary global cache (Project_A) becomes unavailable, the system will automatically fall back to the secondary global cache (Project_B).
          To enable this behavior, redirect the secondary cache configuration to the primary cache configuration and merge the required mount path settings as shown below:
                cat $workdir/.spillbox/sites/local/site_b//terraform/etc/spillbox_config_cache >> $workdir/.spillbox/sites/local/site_a//terraform/etc/spillbox_config_cache     # Redirect secondary cache configuration to primary
                cat $workdir/.spillbox/sites/local/site_a//terraform/../sites/local/customers/mount_paths/PROJECT_A  > $workdir/.spillbox/sites/local/site_a//terraform/../sites/local/customers/mount_paths/user_mount_conf   # Configure primary project mount paths
                cat $workdir/.spillbox/sites/local/site_b//terraform/../sites/local/customers/mount_paths/PROJECT_B >> $workdir/.spillbox/sites/local/site_a//terraform/../sites/local/customers/mount_paths/user_mount_conf   # Append secondary project mount paths for fallback

Ans:    Please email support@spillbox.io.