LSF Emulation

The following information describes the available resources, specific scripts, and guidelines of using those resources.

The Altair Accelerator installation includes scripts that are designed to minimize the impact of transitioning workloads from using LSF directly and instead using the Accelerator Plus hierarchical scheduler. Ideally, the use of these scripts, will enable the redirection of workloads from using an LSF cluster directly to one using an Accelerator Plus scheduler in front of the same LSF cluster. While the scripts cover the common usage of LSF, higher performance and a richer set of scheduling directives will be achieved with using the native Accelerator Plus command set.

The scripts that emulate the above commands are available in the directory $VOVDIR/scripts/lsfemulation, which is not in the PATH in the default setup. This setup avoids collision with the platform scripts and commands.

% ves +LSFEMUL
% bsub sleep 10
% bjobs -a

Configure Resource Mapping

The resources used by LSF, expressed by the -R option in bsub, are significantly different from the resource maps in Accelerator.

To map resources from one system to the other, customize the file $VOVDIR/local/lsfemulation/config.bsub.tcl

An example for config.bsub.tcl can be found in the usual location for configuration files, a subdirectory of $VOVDIR/etc/config:

File: $VOVDIR/etc/config/lsfemulation/config.bsub.tcl

#
# Sample configuration of the bsub emulation.
#
# This file must be placed in $VOVDIR/local/lsfemulation/config.bsub.tcl
#

# If you want support for exclusive access to machines (option -x)
# you need to:
# 1. Uncomment the line below 'set bsubopt(percent) 1'
# 2. Make sure all taskers offer the resource PERCENT/100
# 3. Make sure all jobs request at least PERCENT/1 (see vnc_policy.tcl)
set bsubctrl(percent) 1

# Set this to 1 to cause bsub to always send email
set bsubctrl(alwaysmail) 1

# Truncate emailed log files after this many bytes
# negative values (e.g. -1) mean mail the whole file (BEWARE)
# zero means accept the default (65536 bytes)
set bsubctrl(logmax) 0

# Emulation transforms -m hostnames into an OR expression,
# which can slow down the NC scheduler if too complex.
# Hosts after this count are silently dropped to avoid slow scheduling
set bsubctrl(moptmax) 6

# Map LSF 'select' resources into NC resources.
# select[rhel4]  -> "linux"
set MAP_LSF2NC(rhel3) "linux"
set MAP_LSF2NC(rhel4) "linux"
set MAP_LSF2NC(rhel5) "linux"
set MAP_LSF2NC(RH4_64) "linux x86_64"

# Map LSF 'rusage' resources into NC resources,
# typically resources of type License:
# Example:
#  rusage[dc=1]   -> "License:Design-Compiler"

set MAP_RUSAGE(dc)   "License:Design-Compiler"
set MAP_RUSAGE(pt)   "License:PrimeTime"
set MAP_RUSAGE(drc)  "License:lic_drc"
set MAP_RUSAGE(lvs)  "License:lic_lvs"
set MAP_RUSAGE(erc)  "License:lic_erc"

Emulate the LSF Report in the Output Log

Some legacy scripts expect some LSF specific lines in the log file of a job. This can be achieved with a post-command that adds those lines to the log. An example of such command is post_job_report.sh.

#!/bin/csh -f
# -*- Tcl -*- \
   exec vovsh -f $0 $*:q

set usage "
Description:
  post_job_report.sh

  Used for some jobs submitted with the bsub emulator:

Example:
% ves +LSFEMUL
% bsub -Ep $VOVDIR/etc/post/post_job_report.sh -J test.lsf_with_jobreport cal 2015
"

if { $argv == {} } {
    VovPrintUsage $usage
}

source $env(VOVDIR)/tcl/vtcl/vovlsfemulib.tcl

set jobId       [lindex $argv 0]
set logFileName [lsfEmuGetJobLogFileName $jobId]
set report      [lsfEmuFmtJobReport $jobId]

if { $logFileName ne "" } {
    VovMessage "Adding job report to $logFileName"
    set fp [open $logFileName "a"]
    puts $fp [lsfEmuFmtJobReport $jobId]
    close $fp
} else {
    VovMessage "No log file found for job $jobId\n$report"

    set whyOld ""
    set whyNew "Cannot find a log file for this job $jobId"
    if { [catch {set whyOld [vtk_prop_get $jobId WHY]}] } {
        set why $whyNew
    } else {
        set why "$whyOld\n$whyNew\n$report"
    }
    catch {vtk_prop_set $jobId WHY $why}
}

exit 0

% bsub -Ep $VOVDIR/etc/post/post_job_report.sh   [OTHER OPTIONS]
... command

Debug the LSF Emulation Layer Usage

To debug as well as test and verify an LSF emulation script, it can be helpful to view the issued commands and the used options and values.

% setenv VOV_LOG_LSFEMUL ~/lsfemul.log
% bsub sleep 11
% lsid
% cat $VOV_LOG_LSFEMUL