Environment Management: Limits

In addition to environment variables, hard and soft limits can affect tool behavior. Hard and soft limits are set in the shell and are imposed by the operating system. Both UNIX and Windows provide a mechanism for processes to communicate information to their subprocesses via environment variables.

VOV uses special environment variables to communicate limit information. The environment variables are named VOV_LIMIT_<name>. name is the name of the limit, such as VOV_LIMIT_cputime.

The variables are interpreted in the wrapper program vw2 which uses the C-language getrlimit()/setrlimit() system calls to set the limits for the child process when the job runs.

The value of a variable can be either of the following (depending on the value type):

• A ByteSpec, which is a sequence of digits followed by an optional unit indicator letter, e.g. 5M
• A TimeSpec, which is a sequence of integers and unit indicators, e.g. 3m, or 4h30m
• A sequence of digits without a unit indicator, e.g. 1000 (applicable to all value types)
• The string value 'unlimited' (applicable to all value types)

The recognized unit indicators (case insensitive) for the ByteSpec format are:

• K, kilobytes (same as no unit indicator)
• M, Megabytes (multiply by 1024)
• G, Gigabytes (multiply by 1024*1024)
• T, Terabytes (multiply by 1024*1024*1024)

The recognized unit indicators (case insensitive) for the TimeSpec format are:

• s, Seconds (same as no unit indicator)
• m, Minutes (multiply by 60)
• h, Hours (multiply by 60*60)
• d, Days (multiply by 60*60*24)
• w, Weeks (multiply by 60*60*24*7)
• y, Years (multiply by 60*60*24*365)

% nc run -e 'BASE+D(VOV_LIMIT_cputime=20)' .... shortjob.csh
% nc run -e 'BASE+D(VOV_LIMIT_vmemoryuse=5G)' .... bigjob.csh
% nc run -e 'BASE+D(VOV_LIMIT_vmemoryuse=5G)' .... bigjob.csh

% nc run -e 'BASE+D(VOV_LIMIT_vmemoryuse=5000M)'    .... bigjob.csh
% nc run -e 'BASE+D(VOV_LIMIT_vmemoryuse=5G)'       .... bigjob.csh
% nc run -e 'BASE+D(VOV_LIMIT_vmemoryuse=5000000k)' .... bigjob.csh

The multipliers used with the memory specifications for limits are as follows: 'k' (KiB, 1024=2¹⁰, kibibytes) 'M' (MiB, 2²⁰, mebibytes) 'G' (GiB, 2³⁰, gibibytes) and 'T' (TiB, 2⁴⁰, tebibytes). The multipliers are case-insensitive.

If an integer has no unit specification, kilobytes are the units used. For example, 1024K is the same as 1024. In addition, using unlimited as a value is acceptable.

The following two examples show how to find the current limits.

% limit
cputime         unlimited
filesize        unlimited
datasize        unlimited
stacksize       unlimited
coredumpsize    0 kbytes
vmemoryuse       unlimited
descriptors     1024 
memorylocked    unlimited
maxproc         2048 
openfiles       1024 

bash-2.05$ ulimit -a
core file size (blocks)     unlimited
data seg size (kbytes)      unlimited
file size (blocks)          unlimited
max locked memory (kbytes)  unlimited
max memory size (kbytes)    unlimited
open files                  1024
pipe size (512 bytes)       8
stack size (kbytes)         unlimited
cpu time (seconds)          unlimited
max user processes          5119
virtual memory (kbytes)     unlimited

## Use option -H to get the hard-limits
bash-2.05$ ulimit -a -H         
...

In most cases, everything in the shell startup file can be set as unlimited. This setup gives the tools the greatest possibility of a successful run. It is extremely rare for this method to not work.

When an environment snapshot is used when submitting jobs to Accelerator, the limits in the submission environment are captured automatically.

if { [info command vtk_umask_get] != {} } {
    setenv VOV_UMASK [vtk_umask_get]
    catch {
        vtk_limits_get limit
        foreach n [array names limit] {
            setenv VOV_LIMIT_$n $limit($n)
        }
    }
}

Tcl-language API

VOV provides two API procedures to get and set the limits, vtk_limits_get and vtk_limits_set. Both procedures take a single array parameter, which contains the limits, keyed by name.

Because different platforms have different limits available, the VTK procedures support only a common subset of limits.

catch {
    vtk_limits_get L ;      # get existing limits into array L
    set L(coredumpsize) 0 ; # set coredump limit
    foreach n [array names L] {
        setenv VOV_LIMIT_$n $limit($n) ; # propagate limits to envVars
    }
}