What's New

1. Acquire the Mistral license and place it in $VOVDIR/local as mistral.dat.
2. Run the command nc run in CLI with the -ioprofile option. For example, nc run -ioprofile <job>. To test this you can run a script that reads/writes to a file.
3. In your working directory, the Mistral log file should be generated inside the vnc_logs/<current date> folder
4. Run the command nc gui -ioprofile<jobId> in CLI to get graphical view of the log data. You can also achieve this by running nc gui, right clicking on your job and clicking on View Mistral Profile.

1. If you prefer no confirmation prompt when invoking install.sh to do the CLI installation, specify the -noconfirm option.
2. The behavior with -local link is different with install.sh vs. batch_install.csh. Use the -local linkext with install.sh to get the equivalent of -local link with batch_install.csh. Read the usage syntax information via install.sh -h for full details.

1. Automatic zipping and unzipping feature is implemented for FILEX and PHANTOM databases. Currently FILEX and PHANTOM end up with new FILE versions of the output file, with the old one disconnected. This issue cannot be fixed because it is not feasible to add the database option to the capsule which can avoid creating same file with different database.
2. FDL proc I and O are updated with the -zippable argument.
3. Fixed issue with automatic zipping. It will be triggered in a routine task every 1 minute.
4. Fixed issue with new "zip" jobs getting created over and over.

In Allocator Resources Summary page: Added ordinal number to each row in the first column. The Unassigned row has been fixed to be always the last one. Added ascending and descending sorting. By default, the Nickname column is ordered as per the order in config file.

1. Login to a RHEL 7.9 host or equivalent that has the patchelf utility installed and run the post-installation script: $VOVDIR/../scripts/install-portable-arch.sh.
2. Login to a RHEL 6.10 host or equivalent, set the VOV_PORTABLE_ARCH environment variable to 1, then source the vovrc script for your shell.
3. If using with WX, be sure to add the follwoing line to vovwxd's config.tcl: set CONFIG(exeleaf) "wxagent"

nc run -ioprofile <job> , nc gui -ioprofile <jobId>

• vovslavelaunch
• vovslavelaunchd
• vovslavelist
• vovslaveloadutil
• vovslavemgr
• vovslaveowner
• vovslavestartup

nc cmd vovacl -agent "USER user0001" -set -actions ALL $id"

Implemented reservation property hardfill and softfill to restrict the scheduler to backfill hardfill reservation with only autokill jobs, while softfill reservation may be backfilled with autokill and xdur jobs.

Modified vovtaskermgr utility to support all 4 options.

• vovslavelaunch
• vovslavelaunchd
• vovslavelist
• vovslaveloadutil
• vovslavemgr
• vovslaveowner
• vovslavestartup

• enableBackfillResReservation
• enableLookAheadReservation
• lookAheadSchedInterval
• lookAheadTimeResolution
• maxLookAheadReservationCount
• maxLookAheadSchedulingDuration

If not set, NC_QUEUE will be set on the job environment only if it's explicitly set in the job submission environment and the user is using SNAPPROP or SNAPSHOT.

If wx.setQueueEnv is set to 1 and NC_QUEUE is not set in the submission environment, NC_QUEUE in the job environment will be the WX project name.

If NC_QUEUE is set in the submission environment and SNAPPROP or SNAPSHOT is used, NC_QUEUE will be propagated from the submission environment to the job environment.

1. Resources - Plot: legend will not overlap the following plot. To solve this, will be displayed in more than one column if needed.
2. Overview: resource name and all sites columns are fixed in case there are horizontal scroll because the entries in the table are bigger than the visible screen
3. Overview: table sorting in descending order
4. Resources - Summary: Table sorting in descending order

1. Jobs may appear stopped, but will actually continue to run.
2. A second stop request may cause the tasker to go into a BUSY state for several seconds while the tasker reconciles the fact that the job is still running.

vovserver(54840) ERROR Jun 15 16:22:58 Resource map id not found for resource License:111. 
[mqlib/mqres:596] 
vovserver(54840) ERROR Jun 15 16:22:58 LA:countJobs:Resource License:111 020193489 without a resmap 
[trace/tracelicense:1114]

wx -q $WXQ run -r Limit:@USER@_foo_1 -- sleep 0

1. 1. In the treemap now all rectangles are visible, nothing is cut off.
2. In the treemap legend nothing is cut off, everything is aligned and the records are ordered in decreasing order.

nc cmd vovacl -agent "USER user0001" -set -actions ALL $id"

Implemented reservation property hardfill and softfill to restrict the scheduler to backfill hardfill reservation with only autokill jobs, while softfill reservation may be backfilled with autokill and xdur jobs.

Modified vovtaskermgr utility to support all 4 options.

• vovslavelaunch
• vovslavelaunchd
• vovslavelist
• vovslaveloadutil
• vovslavemgr
• vovslaveowner
• vovslavestartup

• enableBackfillResReservation
• enableLookAheadReservation
• lookAheadSchedInterval
• lookAheadTimeResolution
• maxLookAheadReservationCount
• maxLookAheadSchedulingDuration

If not set, NC_QUEUE will be set on the job environment only if it's explicitly set in the job submission environment and the user is using SNAPPROP or SNAPSHOT.

If wx.setQueueEnv is set to 1 and NC_QUEUE is not set in the submission environment, NC_QUEUE in the job environment will be the WX project name.

If NC_QUEUE is set in the submission environment and SNAPPROP or SNAPSHOT is used, NC_QUEUE will be propagated from the submission environment to the job environment.

source $env(VOVDIR)/tcl/vtcl/vovflexlmdlib.tcl
package require vovurlutils

global HEALTHLIB_PRODUCT_MAP HEALTH_PROCS env

set HEALTHLIB_PRODUCT_MAP(doTestHealthMyFeature) "nc"
# registerHealthCheck doTestHealthMyFeature -checkfreq 10 -forceCheckfreq -mailfreq 10 -forceMailfreq
lappend HEALTH_PROCS(list) doTestHealthMyFeature

proc doTestHealthMyFeature { args } {
    global HEALTH_PROCS
    set homeUrl $HEALTH_PROCS(homeURL)
    # VovMessage "now running doTestHealthMyFeature"
    set subject "TestMyHealthFeature"
    set body "\nHi,\n\nWe ran TestMyHealthFeature.\n"
    doMailNotify doTestHealthMyFeature "@ADMIN" $subject $body
}

TIMEVAR doTestHealthMyFeature {
    Tue {
        suppressMail doTestHealthMyFeature 1
    }
    06:00-08:00 {
        suppressMail ALL 0
        suppressMail doTestHealthMyFeature 0
    }
    default {
        suppressMail ALL 1
        suppressMail doTestHealthMyFeature 1
    }
} 

1. CONFIG(target) - specify the space separated list of cluster(s)/execution host(s), jobs will be submitted using bsub -m option.

   CONFIG(targetName,env) - specify a different named environment per cluster/execution host.

   LSFtarget/LSFTarget - specify cluster/execution host per job in the resource string regardless of the CONFIG(target).

2. CONFIG(cluster) - specify the space-separated list of the cluster(s), jobs will be submitted using bsub -clusters option

   CONFIG(clusterName,env) - specify a different named environment per cluster.

   LSFcluster/LSFCluster - specify cluster per job in the resource string regardless of the CONFIG(cluster).

TIME:STATE:SIGNAL,includerx,excluderx,skiptop 
VovPreemptMethod can now take the form: 
VovPreemptMethod resource SIGNAL -signal SIGNAME -include patternList -exclude patternList 

nc preempt can now take the form: 
nc preempt -method TIME:STATE:SIGNAL,includerx,excluderx,skiptop 

If you reference NC_STOP_SIGNALS, the EXT format is : EXT:SIGNAL:INCLUDERX:EXCLUDERX:SKIPTOP

You can now use :SIGNAL:INCLUDERX:EXCLUDERX:SKIPTOP to do the same thing but without vovjobctrl getting invoked. Job control will be handled directly by the tasker. This also works with VOV_STOP_SIGNALS.

You must set nowrapper=True for a job to be run with no wrapper. When stopping a job via the REST API via a "PUT" request with action='stop', the API now accepts an additional field named "skiptop". If False, the specified stop signal may also be sent to the top job. Normally the top job the is vw wrapper.

When preempting a job via the REST API with a "PUT" request with action='preempt', the API now accepts a syntax for the method field of: TIME:STATE:SIGNAL,includerx,excluderx,skiptop.

This is similar to the updated preemption plan format.

-multiphase [1|0] -mpres "resource string" -multiphase 1 enables multiphase jobs.

-mpres sets the resources that will be used for each phase.

The '%' is used as a delimeter for the resources of each phase. e.g. -mpres "linux64 foo%linux64 bar:linux64 baz" By specifying the resources of each phase and designating that certain resources are only allocated to certain taskers, one can run different phases of a job on different taskers.

vtk_resourcemap_set License:blue UNLIMITED License:blue_tasker1 
vtk_resourcemap_set License:red UNLIMITED License:red_tasker2 
vtk_resourcemap_set License:blue_tasker1 1 tasker1 
vtk_resourcemap_set License:red_tasker2 1 tasker2

nc run -multiphase 1 -mpres "linux64 
License:blue%linux64 License:red%linux64 License:blue" -- -e BASE -D /home/jjmcwill/testDir/testMultistage.sh 

MPRESOURCES: Contains the same resources passed in -mpres, and is used to reset the job resources for each phase.

MPCURRENTPHASE: Contains an integer indicating the current job phase. It starts at one, and has a max value of 9.

What the job script sees: The running job script will see an environment variable named VOV_JOB_PHASE which is set to the current phase. The script writer will need to use that to decide what work to do for that phase. If the script exits with an exit code of 216, Accelerator will increment the job phase, change the job resources, and reschedule the job to run again. If the script exits with an exit code of 0, the job is considered "Done", and MPCURRENTPHASE is reset to 1.

Failed jobs: If a job fails during a phase with a code other than 0 or 216, it is considered FAILED and MPCURRENTPHASE will not increment. If the job is invalided and re-run (e.g. nc rerun -f JOBID), the job will re-run starting at MPCURRENTPHASE and further phases will run if the job exits with code 216, as described above.

Logging: After the first phase is run, subsequent phases of the job will have the command rewritten so that the wrappers are passed "-a -A", telling the wrappers to append to the job log. This is so that all phases of the job get their stdout and stderr logged to the same file. If this was not done, each phase of the job would overwrite the log, and the user would only see the output from the last phase that was run. If Accelerator does not detect one of the standard vov wrappers at the beginning of the command line, it will assume the command is not using a wrapper. In this case, it will look for the standard ">;" redirect symbol in the command and replace it with ">>;".

REST Support: In the payload for submitting a job via rest, two new fields are allowed: multiphase and mpres. Setting multiphase = True enables multiphase job support. Setting the mpres field behaves the same as described for the command line argument described above. Re-running a multiphase job that has failed via the REST re-run API will behave similarly to rerunning a failed multiphase job from the command line as described above.

This has been changed so that the queueName also contains a status of whether or not vovwxd sees the status of the basequeue as ONLINE or DOWN.

The new format of EXTLINKS is: "queueName1:status=STAT1 URL1 queueName2:status=STAT2 URL2 ..."

When visiting the web UI for the wx project, the link for a base queue that is down will be shown in red instead of the customary gray.

BUCKETINFO(AUXRESOURCES)

BUCKETINFO(BUCKETID)

BUCKETINFO(COUNT)

BUCKETINFO(FSGROUP)

BUCKETINFO(JOBCLASS)

BUCKETINFO(JOBPROJ)

BUCKETINFO(OSGROUP)

BUCKETINFO(PRIORITY)

BUCKETINFO(RESOURCES)

BUCKETINFO(USER)

BUCKETINFO(XDUR)

BUCKETINFO(maxidle)

BUCKETINFO(maxlife)

BUCKETINFO(taskerAutokill)

BUCKETINFO(taskerCmd)

BUCKETINFO(taskerReservation)

BUCKETINFO(taskerResources)

The user can set values in vovaccel.tcl.

http.minSSLVersion - set the minimum TLS version supported. Defaults to "TLSv1.3". Valid values are TLSv1, TLSv1.1, TLSv1.2, and TLSv1.3

http.maxSSLVersion - set the maximum TLS version supported. Defaults to "TLSv1.3". Valid values are TLSv1, TLSv1.1, TLSv1.2, and TLSv1.3

http.tls12Ciphers - this is a list of ciphers that TLS 1.2 and earlier will be restricted to. The default is an empty string, but it can be set to a string such as "ECDH+AESGCM:ECDH+AES256:ECDH+AES128:DH+3DES:!ADH:!AECDH:!MD5"

For an explanation of the string format, see "Cipher List Format" at https://www.openssl.org/docs/manmaster/man1/openssl-ciphers.html for details on what's acceptable.

http.tls13Ciphers - This is a list of cipher SUITES that is used by TLS 1.3 (and presumably above if newer TLS versions are introduced in the future) See the description of SSL_set_ciphersuites() at https://www.openssl.org/docs/manmaster/man3/SSL_CTX_set_cipher_list.html for an explanation.

By default the value is an empty string, which tells OpenSSL to use the default, which according to the documentation is TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256:TLS_AES_128_GCM_SHA256"

Changing any four of these values via vovservermgr config while the server is running will cause the internal webserver to stop and restart with the new configuration. We recommend using a tool such as sslscan that is part of RHEL 8 to verify your changes.

1. The filter is most useful to sort out the jobs/Files. So, it has been moved to down to just above the Jobs/Files table and cause it to search only within the Jobs/Files list.
2. The "Jobs" list has been renamed to "Jobs and Files" because sets contain jobs and files.
3. Added the count of items (both the Jobs and Files) in the "Jobs and Files" list. eg: "Jobs and Files ( 237 )".
4. For files in the Jobs and Files table, omit the Status column has been omitted (leave it blank) and fix has been added to the "Type" column to say "File" instead of the incorrect "Job."

• "never" when times are:
  • specially formatted patterns
  • over 12/31/2099 in the future
  • causing errors when processing
• "expired" - all times are in the past
• <Actual Date> - earliest expiration time in the future

Will NOT show: "some expired" - deprecated

1. When using the tasker configuration path (vtk_tasker_define), ensure that the correct tasker-based procedures are defined and called.
2. Define NC_QUEUE in the indirect tasker's execution environment using the queue name that is specified in the vtk_tasker_nc call, which removes the requirement to create a taskerVNC.tcl in $VOVDIR/local that defines it.
3. Fix major efficiency issue that prevented the indirect tasker from notifying the server that it is ready to accept another job once one has finished.
4. Improve messaging to display the queue name that is being interfaced with for debuggability purposes.

1. Check for the presence of a license key file at server startup so that the default enterprise license mode will be "Off" if a key file is found and the policy.tcl file does not have an explicit setting for the mode.
2. Prevent the above check upon every request to check licensing so that the mode doesn't switch unless manually requested.
3. Allow for switching the mode out of "Off".
4. Restore the mode control and license request sections in license.cgi when the mode is "Off".
5. Rename the mode selection in the license.cgi drop-down menu to "Off (use license key file)" so that the actual value is represented along with the intent.

• The Taskerlists now contain the tasker names instead of only NC-internal IDs. As a result, the taskerlist does not have te be recreated when a tasker is stopped and then restarted, as had been the case with slavelists in previous releases.
• The TaskerLists definition will be stored in the PR so they will survive vovserver stop-and-restart sequences.
• Added support for resource 'TaskerList:XXX' in nc run
• SlaveList:XXX' resources will be automatically replaced by 'TaskerList:XXX' resources.

1. Fixed an issue where the NUMA_AFFINITY property on jobs always showed it as being bound to specific CPUs, even though it was bound to the entire NUMA node.
2. Fixed an issue where WX vovtaskers were overriding the affinity set by the NC vovtasker, causing WX jobs to be bound to only a subset of the available CPUs.

1. Added the following headers to all HTTP responses:
   • Strict-Transport-Security
   • X-Frame-Options
   • X-Xss-Protection
   • X-Content-Type-Options
   • X-Content-Security-Policy
   • Content-Security-Policy
2. Added support for web ports higher than 32000.

1. The VOV_CONTAINER_<resource> (where <resource> is CORES, RAM, or TMP) environment variable was not being set in the hook environments for containers that were configured to use the job's resource specification as container limits.
2. The VOV_CONTAINER_JOB_RUNDIR environment variable was not being set in the hook environments if the job's run directory had been configured as an equivalence, as would be the case for jobs submitted from within the user's home directory.

set ::jobname_lexicon legacy

set ::jobname_lexicon replace

• The option -resumedelay default is now 5 seconds previously the CLI default was 10 seconds.
• The default pool name is now mainpool, previously the web UI default was MainPool.
• The default for -mqthresh is now 0.1=10%, before the web UI default was 0.01.
• The -killage option default is now 60, previous version of the web UI had a default of 0.
• For -skipresumedjob option default is 120, previous versions of the web UI had a default of 0.
• For -preemptslavenum new default will always be 1, some variants of CLI had a default value of 2 previously.

Licensing robustness and network fault-tolerance have been significantly improved. The software can operate in "disconnect mode", which means that it connects, gets a checkout, then disconnects from the license server. Checkout is refreshed every hour. The software will not fail for until 5 full days have passed since the last successful checkout. Alerts are issued for this as well.

As before, license servers may be specified as a colon-separated list via the RMonitor_LICENSE environment variable