Slurm Jobs

By default, Slurm does not source the files ~./bashrc or ~/.profile when requesting resources via sbatch.  If your job submission requires environment settings specified in your .bashrc, be sure to include the following in your job submission script after the module loads:

source ~/.bashrc

The three most important commands in Slurm are sbatch, srun and scancel. sbatch is used to submit a job script to the queue like the one below, called example. sbatch srun is used to run parallel jobs on compute nodes. Jobs can be canceled with scancel. To see template job submission script for different software package including Ansys, Abaqus, Matlab, CP2K, etc., please take a look at the following directory on the cluster: /ihome/crc/how_to_run/

#!/bin/bash
#SBATCH --job-name=<job_name>
#SBATCH --nodes=<number of nodes> #number of nodes requested
#SBATCH --ntasks-per-node=1
#SBATCH --cluster=mpi # mpi, gpu and smp are available in H2P
#SBATCH --partition= # available: smp, high-mem, opa, gtx1080, titanx, k40
#SBATCH --mail-user=<user_ID>@pitt.edu #send email to this address if ...
#SBATCH --mail-type=END,FAIL # ... job ends or fails
#SBATCH --time=6-00:00:00 # 6 days walltime in dd-hh:mm format
#SBATCH --qos=long # required if walltime is greater than 3 days
module purge #make sure the modules environment is sane
module load intel/2017.1.132 intel-mpi/2017.1.132 fhiaims/160328_3
cp <inputs> $SLURM_SCRATCH # Copy inputs to scratch
cd $SLURM_SCRATCH
# Set a trap to copy any temp files you may need
run_on_exit(){
 cp -r $SLURM_SCRATCH/* $SLURM_SUBMIT_DIR
}
trap run_on_exit EXIT 
srun <job executable with parameters> # Run the job
crc-job-stats.py # gives stats of job, wall time, etc.
cp <outputs> $SLURM_SUBMIT_DIR # Copy outputs to submit directory