Manual Preemption

Manual preemption can be used in addition to or instead of automatic preemption.

In automatic preemption, the preemptable jobs are identified by vovpreemptd; however, in manual preemption, the ID of the preemptable job is required with the nc preempt command.

It is sometimes advantageous to manually preempt a job. For example, the licenses are available, but all the CPU slots are taken by other jobs and an important job must run now. Preempting a job can only be applied by the owner of the job or by having ADMIN privileges.

A job can be submitted with very high priority, such as high or top. Later, if needed, a running job can be preempted with nc preempt jobId if the high level job is waiting in the queue.

Unless the method is specified with the -method, manual preemption uses the configured preemption method for the resources held by the preempted jobs, which is the same as applied for automatic preemption. The preemption method is defined in the configuration file for the automatic preemption daemon. See Automatically Start the Preemption Daemon for the location of this file.

Examples:

% nc preempt 34567
% nc preempt -manualresume 45678

Manual Preemption with Manual Resumption

In normal preemption, the preempted job is suspended and another job, called the resumer job, is created. The resumer job is scheduled to be executed as soon as the required licenses become available.

% nc preempt 12345
### No need to call nc resume

With the option -manualresume of nc preempt, the resumer job is created but not scheduled. To schedule the resumer job, use nc resume.

% nc preempt -manualresume 12345
# ... later, you have to remember to resume the job...
% nc resume 12345

nc preempt

Preempt the specified running jobs.


nc: Usage Message

NC PREEMPT:
    Preempt the specified running jobs.
    Preemption means that:
    1.  The job is stopped or suspended (depending on method and age)
    2.  The resources of the job are revoked
    3.  If needed, a 'resumer job' is scheduled to
    restart the job as soon as the revoked
    resources are again available.

    If a job is not running, an error is reported.

USAGE:
    % nc preempt [OPTIONS] <jobId> ...

OPTIONS:
    -h                   -- This help.
    -v                   -- Increase verbosity.
    -method METHOD       -- Specify preemption method. Default is AUTOMATIC.
                            Common values:
                            AUTOMATIC, KILL, KILL+RESUBMIT, SUSPEND,
                            SIGTSTP+SUSPEND
                            Other values and example of preemption plans
                            (see docs): MODELSIM, BEGIN:RETRACING:EXT,
                            TSTP, vish.
                            Bad values are currently ignored.
    -manualresume        -- The resumer job is not scheduled;
                            A 'nc resume' is required to restart the
                            preempted job.
    -resumeres RESLIST   -- Specify resources to be added to the
                            resumer job. The resources are expanded.
                            Examples:
                            @HOST@ RAM/@RAM@
                            @PROP.SOLUTION@
                            Any job field can be used, but here are
                            some common fields that can be useful:
                            @TASKERNAME@
                            @HWRAM@ @HWCORES@ @HWPERCENT@ @HWSLOTS@

EXAMPLES:
    % nc preempt 123456
    % nc preempt -v 123456

    % nc preempt -method SIGTSTP 123456
    % nc preempt -method KILL+RESUBMIT 123456
    % nc preempt -method BEGIN:RETRACING:EXT,TSTP,vish 123456
    % nc preempt 03076307 03076311 03076315

    % nc preempt -manualresume 123456
    % nc preempt -manualresume -resumeres @PROP.SOLUTION@ 123456
    % nc preempt -manualresume -resumeres "@HOST@ RAM/@MAXRAM@" 123456
    % ...
    % nc resume  123456

Options

Most of the options are explained by the summaries in the brief usage output shown above. The -v option enables printing of additional messages that may be helpful in troubleshooting.

The -method option may be used to specify the preemption method used to revoke the resources of the preempted jobs. This overrides any method in the configuration file. This option may be helpful in troubleshooting. The -l option is handy to get a list of known preemption methods. The names of the methods are case-sensitive.

Only users with the Altair Accelerator ADMIN privilege or the owner of the preempted job can run this command.