-R

A resource requirement string describes the resources a job needs. LSF uses resource requirements to select hosts for job execution. Resource requirement strings can be simple (applying to the entire job), compound (applying to the specified number of slots), or alternative.

select[selection_string] order[order_string] rusage[
usage_string [, usage_string][|| usage_string] ...]
span[span_string] same[same_string] cu[cu_string]] affinity[affinity_string]

The square brackets must be typed as shown for each section. A blank space must separate each resource requirement section.

You can omit the select keyword and the square brackets, but the selection string must be the first string in the resource requirement string. If you do not give a section name, the first resource requirement string is treated as a selection string (select[selection_string]).

For example:

bsub -R "type==any order[ut] same[model] rusage[mem=1]" myjob

is equivalent to the following:

bsub -R "select[type==any] order[ut] same[model] rusage[mem=1]" myjob

The size of the resource requirement string cannot exceed 512 characters. If you need to include a hyphen (-) or other non-alphabet characters within the string, enclose the text in single quotation marks, for example, bsub -R "select[hname!='host06-x12']".

If LSF_STRICT_RESREQ=Y in lsf.conf, the selection string must conform to the stricter resource requirement string syntax described in Administering IBM Platform LSF. The strict resource requirement syntax only applies to the select section. It does not apply to the other resource requirement sections (order, rusage, same, span, or cu). When LSF_STRICT_RESREQ=Y in lsf.conf, LSF rejects resource requirement strings where an rusage section contains a non-consumable resource.

If RESRSV_LIMIT is set in lsb.queues, the merged application-level and job-level rusage consumable resource requirements must satisfy any limits set by RESRSV_LIMIT, or the job will be rejected.

Any resource for run queue length, such as r15s, r1m or r15m, specified in the resource requirements refers to the normalized run queue length.

By default, memory (mem) and swap (swp) limits in select[] and rusage[] sections are specified in KB. Use LSF_UNIT_FOR_LIMITS in lsf.conf to specify a larger unit for these limits (MB, GB, TB, PB, or EB).

bsub -R "sol10 || sol11" myjob

bsub -R "swp > 15 && hpux order[ut]" myjob

bsub also accepts multiple -R options for the order, same, rusage (not multi-phase), and select sections. You can specify multiple strings instead of using the && operator:

bsub -R "select[swp > 15]" -R "select[hpux] order[r15m]" -R rusage[mem=100]" -R "order[ut]" -R "same[type]" -R "rusage[tmp=50:duration=60]" -R "same[model]" myjob

When application-level, and queue-level cu sections are also defined, the job-level cu section takes precedence and overwrites both the application-level and queue-level requirement definitions.

bsub -n 64 -R "cu[excl:type=enclosure:maxcus=4]" myjob

bsub -R "bigmem" myjob

bsub -R "defined(bigmem)" myjob

bsub -R "select[defined(verilog_lic)] rusage[verilog_lic=1]" myjob

bsub -R "rusage[mem=20, license=1:duration=2]" myjob

bsub -R "rusage[mem=20:swp=50:duration=1h, license=1:duration=2]" myjob

bsub -R "rusage[mem=20,swp=50:duration=1h, license=1:duration=2]" myjob

bsub -R "rusage[swp=50:duration=2h:decay=1, license=1:duration=2]" myjob

bsub -R "rusage[mem=20:duration=30:decay=1, lic=1:duration=30]" myjob

bsub -R "rusage[mem=(50 10):duration=(10):decay=(0)]" myjob

You are running an application version 1.5 as a resource called app_lic_v15 and the same application version 2.0.1 as a resource called app_lic_v201. The license key for version 2.0.1 is backward compatible with version 1.5, but the license key for version 1.5 does not work with 2.0.1.

Job-level resource requirement specifications that use the || operator take precedence over any queue-level resource requirement specifications.

• If you can only run your job using one version of the application, submit the job without specifying an alternative resource. To submit a job that only uses app_lic_v201:

  bsub -R "rusage[app_lic_v201=1]" myjob

• If you can run your job using either version of the application, try to reserve version 2.0.1 of the application. If it is not available, you can use version 1.5. To submit a job that tries app_lic_v201 before trying app_lic_v15:

  bsub -R "rusage[app_lic_v201=1||app_lic_v15=1]" myjob

• If different versions of an application require different system resources, you can specify other resources in your rusage strings. To submit a job that uses 20 MB of memory for app_lic_v201 or 20 MB of memory and 50 MB of swap space for app_lic_v15:

  bsub -R "rusage[mem=20:app_lic_v15=1||mem=20:swp=50:app_lic_v201=1]" myjob

You can specify a threshold at which the consumed resource must be at before an allocation should be made. For example:

bsub -R "rusage[bwidth=1:threshold=5]" myjob

A job is submitted that consumes 1 unit of bandwidth (the resource bwidth), but the job should not be scheduled to run unless the bandwidth on the host is equal to or greater than 5.

In this example, bwidth is a decreasing resource and the threshold value is interpreted as a floor. If the resource in question was increasing, then the threshold value would be interpreted as a ceiling.

An affinity resource requirement string specifies CPU and memory binding requirements for a resource allocation that is topology aware. An affinity[] resource requirement section controls the allocation and distribution of processor units within a host according to the hardware topology information that LSF collects.

Compound resource requirements

In some cases different resource requirements may apply to different parts of a parallel job. The first execution host, for example, may require more memory or a faster processor for optimal job scheduling. Compound resource requirements allow you to specify different requirements for some slots within a job in the queue-level, application-level, or job-level resource requirement string.

Compound resource requirement strings can be set by the application-level or queue-level RES_REQ parameter, or used with bsub -R when a job is submitted. bmod -R accepts compound resource requirement strings for pending jobs but not running jobs.

Special rules take effect when compound resource requirements are merged with resource requirements defined at more than one level. If a compound resource requirement is used at any level (job, application, or queue) the compound multi-level resource requirement combinations apply.

Compound resource requirement strings are made up of one or more simple resource requirement strings as follows:

num1*{simple_string1} + num2*{simple_string2} + ...

where numx is the number of slots affected and simple_stringx is a simple resource requirement string.

The same resource requirement can be used within each component expression (simple resource requirement). For example, suppose static strings resource res1 and res2 are defined. We permit a resource requirement such as:

"4*{select[io] same[res1]} + 4*{select[compute] same[res1]}"

With this resource requirement, there are two simple subexpressions, R1 and R2. For each of these subexpressions, all slots must come from hosts with equal values of res1. However, R1 may occupy hosts of a different value than those occupied by R2.

You can specify a global same requirement that takes effect over multiple subexpressions of a compound resource requirement string. For example,

"{4*{select[io]} + 4*{select[compute]}} same[res1]"

This syntax allows users to express that both subexpressions must reside on hosts that have a common value for res1.

In general, there may be more than two subexpressions in a compound resource requirement. The global same will apply to all of them.

Arbitrary nesting of brackets is not permitted. For example, you cannot have a global same apply to only two of three subexpressions of a compound resource requirement. However, each subexpression can have its own local same as well as a global same for the compound expression as a whole. For example, the following is permitted:

"{4*{same[res1]} + 4*{same[res1]}} same[res2]"

In addition, a compound resource requirement expression with a global same may be part of a larger alternative resource requirement string.

A compound resource requirement expression with a global same can be used in the following instances:

• Submitting a job: bsub -R "rsrc_req_string" <other_bsub_options> a.out
• Configuring application profile (lsb.applications): RES_REQ = "rsrc_req_string"
• Queue configuration (lsb.queues): RES_REQ = "rsrc_req_string"

Syntax:

• A single compound resource requirement:

  "{ compound_rsrc_req } same[ same_str ]"

• A compound resource requirement within an alternative resource requirement:

  "{{ compound_rsrc_req } same[ same_str ]} || {R}"

• A compound resource requirement within an alternative resource requirement with delay:

  "{R} || {{ compound_rsrc_req } same[ same_str ]}@D"

  where D is a positive integer.

For jobs without the number of total slots specified using bsub -n, the final numx can be omitted. The final resource requirement is then applied to the zero or more slots not yet accounted for as follows:

• (final res_req number of slots) = (total number of job slots)-(num1+num2+ ...)

For jobs with the total number of slots specified using bsub -n num_slots, the total number of slots must match the number of slots in the resource requirement as follows:

• num_slots=(num1+num2+num3+ ...)

For jobs with the minimum and maximum number of slots specified using bsub -n min, max , the number of slots in the compound resource requirement must be compatible with the minimum and maximum specified.

You can specify the number of slots or processors through the resource requirement specification. For example, you can specify a job that requests 10 slots or processors: 1 on a host that has more than 5000 MB of memory, and an additional 9 on hosts that have more than 1000 MB of memory:

bsub -R "1*{mem>5000} + 9*{mem>1000}" a.out

Alternative resource requirements

In some circumstances more than one set of resource requirements may be acceptable for a job to be able to run. LSF provides the ability to specify alternative resource requirements.

An alternative resource requirement consists of two or more individual simple or compound resource requirements. Each separate resource requirement describes an alternative. When a job is submitted with alternative resource requirements, the alternative resource picked must satisfy the mandatory first execution host. If none of the alternatives can satisfy the mandatory first execution host, the job will PEND.

Alternative resource requirement strings can be specified at the application-level or queue-level RES_REQ parameter, or used with bsub -R when a job is submitted. bmod -R also accepts alternative resource requirement strings for pending jobs.

The rules for merging job, application, and queue alternative resource requirements are the same as for compound resource requirements.

If a job with alternative resource requirements specified is re-queued, it will have all alternative resource requirements considered during scheduling. If a @D delay time is specified, it is interpreted as waiting, starting from the original submission time. For a restart job, @D delay time starts from the restart job submission time.

An alternative resource requirement consists of two or more individual resource requirements. Each separate resource requirement describes an alternative. If the resources cannot be found that satisfy the first resource requirement, then the next resource requirement is tried, and so on until the requirement is satisfied.

Alternative resource requirements are defined in terms of a compound resource requirement, or an atomic resource requirement:

bsub -R “{C1 | R1 } || {C2 | R2 }@D2 || ... || {Cn | Rn }@Dn” 

Where:

• || separates one alternative resource from the next
• C is a compound resource requirement
• R a resource requirement which is the same as the current LSF resource requirement, except when there is:
  • No rusage OR (||).
  • No compute unit requirement cu[...]
• D is a positive integer:
  • @D is optional: Do not evaluate the alternative resource requirement until 'D' minutes after submission time, and requeued jobs still use submission time instead of requeue time. There is no D1 because the first alternative is always evaluated immediately.
  • D2 <= D3 <= ... <= Dn
  • Not specifying @D means that the alternative will be evaluated without delay if the previous alternative could not be used to obtain a job's allocation.

For example, you may have a sequential job, but you want alternative resource requirements (that is, if LSF fails to match your resource, try another one).

bsub -R "{ select[type==any] order[ut] same[model] rusage[mem=1] } ||  
{ select[type==any] order[ls] same[ostype] rusage[mem=5] }" myjob

You can also add a delay before trying the second alternative:

bsub -R "{ select[type==any] order[ut] same[model] rusage[mem=1] } || 
{ select[type==any] order[ls] same[ostype] rusage[mem=5] }@4" myjob

You can also have more than 2 alternatives:

bsub -R "{select[type==any] order[ut] same[model] rusage[mem=1] } || 
{ select[type==any] order[ut] same[model] rusage[mem=1] } || 
{ select[type==any] order[ut] same[model] rusage[mem=1] }@3 || 
{ select[type==any] order[ut] same[model] rusage[mem=1] }@6" myjob

Some parallel jobs might need compound resource requirements. You can specify alternatives for parallel jobs the same way. That is, you can have several alternative sections each with brace brackets ({ }) around them separated by ||):

bsub -n 2 -R "{ 1*{ select[type==any] order[ut] same[model] rusage[mem=1]} + 1
*{ select[type==any] order[ut] same[model] rusage[mem=1] } } ||
{  1*{ select[type==any] order[ut] same[model] rusage[mem=1]} + 
1*{ select[type==any] order[ut] same[model]
rusage[mem=1] } }@6" myjob

Alternatively, the compound resource requirement section can have both slots requiring the same resource:

bsub -n 2 -R "{ 1*{ select[type==any] order[ut] same[model] rusage[mem=1]} 
+1*{ select[type==any] order[ut] same[model] rusage[mem=1] } } || 
{  2*{ select[type==any] order[ut] same[model] rusage[mem=1] } }@10" myjob 

An alternative resource requirement can be used to indicate how many tasks the job requires. For example, a job may request 4 tasks on Solaris host types, or 8 tasks on Linux86 hosts types. If the -n parameter is provided at the job level then the values specified must be consistent with the values implied by the resource requirement string:

bsub -R " {8*{type==LINUX86}} || {4*{type==SOLARIS}}" a.out

If they conflict, the job submission will be rejected. For example:

bsub -n 3 -R " {8*{type==LINUX86}} || {4*{type==SOLARIS}}" a.out