Slurm QoS 使用

Slurm QoS 介绍

QoS 优先级

如果在 Partition 的 QOS 和 Job 的 QOS 中都设置了限制，Partition 的 QOS 优先，除非 Job 的 QOS 设置了 OverPartQOS 标志时，Job 的 QOS 优先。优先级如下：

Partition QOS limit
Job QOS limit
User association
Account association(s), ascending the hierarchy
Root/Cluster association
Partition limit
None

注意：如果在这个层次结构中有多个点定义了限制，那么列表中第一个定义限制将生效。示例:

MaxJobs=20, MaxSubmitJobs在 Partition QOS 中未定义
Job QOS 和中没有设置限制
MaxJobs=4 和 MaxSubmitJobs=50 在 User association 中定义

生效的限制为：MaxJobs=20 和 MaxSubmitJobs=50

特例：

配置 PartitionTimeLimit 如果设置了，使用此QOS的作业将能够覆盖请求分区的 TimeLimit（如 MaxWallDurationPerJob）。

示例

account 指定 qos 的限制

    
      
      
    
    bash
  
# 创建 account
[root@sms1 ~]# sacctmgr add account xiexianbin Description="xiexianbin" Organization=xiexianbin
 Adding Account(s)
  xiexianbin
 Settings
  Description     = xiexianbin
  Organization    = xiexianbin
 Associations
  A = xiexianbin C = poc
 Settings
  Parent        = root
Would you like to commit changes? (You have 30 seconds to decide)
(N/y): y

# 创建 qos
[root@sms1 ~]# sacctmgr add qos xiexianbin_qos
 Adding QOS(s)
  xiexianbin_qos
 Settings
  Description    = xiexianbin_qos
Would you like to commit changes? (You have 30 seconds to decide)
(N/y): y

# 更新 QoS 配置
[root@sms1 ~]# sacctmgr modify qos xiexianbin_qos set MaxJobsPerUser=256
 Modified qos...
  xiexianbin_qos
Would you like to commit changes? (You have 30 seconds to decide)
(N/y): y
[root@sms1 ~]# sacctmgr modify qos xiexianbin_qos set MaxTRESPerUser=CPU=128
 Modified qos...
  xiexianbin_qos
Would you like to commit changes? (You have 30 seconds to decide)
(N/y): y
[root@sms1 ~]# sacctmgr modify qos xiexianbin_qos set MaxJobsPU=1024
 Modified qos...
  xiexianbin_qos
Would you like to commit changes? (You have 30 seconds to decide)
(N/y): y
[root@sms1 ~]# sacctmgr modify qos xiexianbin_qos set MaxSubmitJobsPerUser=2
 Modified qos...
  xiexianbin_qos
Would you like to commit changes? (You have 30 seconds to decide)
(N/y): y

# 查看 QoS
[root@sms1 ~]# sacctmgr show qos xiexianbin_qos
      Name   Priority  GraceTime    Preempt   PreemptExemptTime PreemptMode                                    Flags UsageThres UsageFactor       GrpTRES   GrpTRESMins GrpTRESRunMin GrpJobs GrpSubmit     GrpWall       MaxTRES MaxTRESPerNode   MaxTRESMins     MaxWall     MaxTRESPU MaxJobsPU MaxSubmitPU     MaxTRESPA MaxJobsPA MaxSubmitPA       MinTRES
---------- ---------- ---------- ---------- ------------------- ----------- ---------------------------------------- ---------- ----------- ------------- ------------- ------------- ------- --------- ----------- ------------- -------------- ------------- ----------- ------------- --------- ----------- ------------- --------- ----------- -------------
xiexianbi+          0   00:00:00                                    cluster                                                        1.000000                                                                                                                                      cpu=128      1024           2

# 绑定 user 和 account
[root@sms1 ~]# sacctmgr add user xiexianbin Account=xiexianbin QOS=xiexianbin_qos Cluster=poc
 Adding User(s)
  xiexianbin
 Associations =
  U = xiexianbi A = xiexianbin C = poc
 Non Default Settings
  QOS           = xiexianbin_qos
Would you like to commit changes? (You have 30 seconds to decide)
(N/y): y

    
      
      
    
    text
  
# 更新 user 的限制
sacctmgr modify user xiexianbin set MaxJobs=2
sacctmgr modify user xiexianbin set MaxSubmitJobs=2
sacctmgr modify user xiexianbin set MaxCPUs=1

# 查看信息
[root@sms1 ~]# sacctmgr show user
      User   Def Acct     Admin
---------- ---------- ---------
      root       root Administ+
xiexianbin xiexianbin      None
[root@sms1 ~]# sacctmgr show account
   Account                Descr                  Org
---------- -------------------- --------------------
      root default root account                 root
xiexianbin           xiexianbin           xiexianbin
[root@sms1 ~]# sacctmgr show Association
   Cluster    Account       User  Partition     Share   Priority GrpJobs       GrpTRES GrpSubmit     GrpWall   GrpTRESMins MaxJobs       MaxTRES MaxTRESPerNode MaxSubmit     MaxWall   MaxTRESMins                  QOS   Def QOS GrpTRESRunMin
---------- ---------- ---------- ---------- --------- ---------- ------- ------------- --------- ----------- ------------- ------- ------------- -------------- --------- ----------- ------------- -------------------- --------- -------------
       poc       root                               1                                                                                                                                                             normal
       poc       root       root                    1                                                                                                                                                             normal
       poc xiexianbin                               1                                                                                                                                                             normal
       poc xiexianbin xiexianbin                    1                                                                                                                                                     xiexianbin_qos

QoS 优先级

示例

account 指定 qos 的限制

参考