Service Units

One service unit (SU) is approximately equal to 1 core hour of computing. The charge is calculated based on factors that the slurm workload manager calls 'Trackable RESources' or TRES. The important TRES values for calculating CRC SUs are:

  • Number of cores requested
  • RAM requested
  • On the GPU cluster, number of cards requested.

Each of these has a TRES Billing Weight assigned to it in the cluster configuration files. These weights along with the amount of resources your job is allocated are used to construct a total cost in SUs.  Here is a table listing the weights for each cluster and partition:

Cluster Partition CPU/GPU Weight Memory Weight
SMP      
  smp 0.8 0.102
  high-mem 1.0 0.0477
MPI      
  opa-high-mem 1 0.149
  mpi 1 0.93
GPU      
  gtx1080 1 0
  v100 5 0
  power9 5 0
  a100 8 0
  a100_multi 8 0
  a100_nvlink 8 0
HTC      
  htc 1 0.128

    To see a more detailed view of this information (including investment hardware configurations), you can use the scontrol slurm command, providing the -M flag to specify a cluster:

    [nlc60@login0b ~] : scontrol -M htc show partition
    PartitionName=htc
       AllowGroups=ALL AllowAccounts=ALL AllowQos=short,normal,long,htc-htc-s,htc-htc-n,htc-htc-l,htc-htc-s-invest,htc-htc-n-invest,htc-htc-l-invest
       AllocNodes=ALL Default=YES QoS=N/A
       DefaultTime=NONE DisableRootJobs=NO ExclusiveUser=NO GraceTime=0 Hidden=NO
       MaxNodes=1 MaxTime=UNLIMITED MinNodes=0 LLN=NO MaxCPUsPerNode=UNLIMITED
       Nodes=htc-1024-n[0-3],htc-n[24-49]
       PriorityJobFactor=1 PriorityTier=1 RootOnly=NO ReqResv=NO OverSubscribe=NO
       OverTimeLimit=NONE PreemptMode=CANCEL
       State=UP TotalCPUs=1792 TotalNodes=30 SelectTypeParameters=NONE
       JobDefaults=(null)
       DefMemPerNode=UNLIMITED MaxMemPerNode=UNLIMITED
       TRES=cpu=1792,mem=19587200M,node=30,billing=2448
       TRESBillingWeights=CPU=1.0,Mem=0.128G
    
    PartitionName=scavenger
       AllowGroups=ALL AllowAccounts=ALL AllowQos=short,normal,long,htc-scavenger-s,htc-scavenger-n,htc-scavenger-l
       AllocNodes=ALL Default=NO QoS=N/A
       DefaultTime=NONE DisableRootJobs=NO ExclusiveUser=NO GraceTime=0 Hidden=NO
       MaxNodes=1 MaxTime=UNLIMITED MinNodes=0 LLN=NO MaxCPUsPerNode=UNLIMITED
       Nodes=htc-n[24-31]
       PriorityJobFactor=1 PriorityTier=1 RootOnly=NO ReqResv=NO OverSubscribe=NO
       OverTimeLimit=NONE PreemptMode=CANCEL
       State=UP TotalCPUs=384 TotalNodes=8 SelectTypeParameters=NONE
       JobDefaults=(null)
       DefMemPerNode=UNLIMITED MaxMemPerNode=UNLIMITED
       TRES=cpu=384,mem=6180000M,node=8
       TRESBillingWeights=CPU=0.0,Mem=0.0G
    

    For a concise view of the TRES used by a job, you can use the sacct command:

    [nlc60@login0b ~] : sacct -X -M smp -j 6169876 --format=User,JobID,Jobname,AllocTRES%30,Elapsed
         User        JobID    JobName                      AllocTRES    Elapsed
    --------- ------------ ---------- ------------------------------ ----------
        nlc60 6169876      hello_wor+         cpu=1,mem=4018M,node=1   00:00:01