Upgrading to 2019.9 and Beyond

Changes to Job State Transition Model

Altair FlowTracer 2015.09 introduced a major change in the state transition model for jobs in a flow. This change in the model will affect existing scripts that have code which depends on details of the earlier model.

In earlier versions, a work task might be represented by more than one job node in the flow. This would happen when a job node reached the RETRACING state. This would cause a new related job node to be created in a RUNNING state with a different job ID. This new job node in the RUNNING state represented the same work task. The two nodes with different node IDs and different state would exist to represent the work task. If the work task successfully completed, then the original job node in RETRACING state would vanish and the new job node that was in RUNNING state would continue to exist to represent the work task. If the work task failed, then the new job node in the RUNNING state would vanish and the original job node would remain to represent the work task.

With Altair FlowTracer 2015.09, the state transition model changed so that there is only one job node to represent the work task. It maintains the same job ID over the entire life of the work task. When a given job reaches the RETRACING state, it then changes to a RUNNING state, and from there to the other states in its future. No related job nodes are created to represent the work task.

Script Changes

Existing scripts may depend on the earlier state model while checking the state of the system. Below are some common cases with suggestions on how to change the scripts to fit the new model.

RETRACINGID

Anything that asks for the RETRACINGID of a job is no longer valid. A request for RETRACINGID simply returns a zero.

set runningJobId30 [vtk_node_format $jobId30 "@RETRACINGID@"]

New:

Delete all the code that uses $runningJobId30 if it no longer has meaning, or use @ID@ instead of @RETRACINGID@".

Counting Running Jobs

In the old model, finding the count of running jobs could be done simply by looking for job nodes in the RETRACING state. In the new model, finding the count requires looking for job nodes in both RETRACING and RUNNING state.

set count_running = vovset count JOBRUNNINGDURINGCRASH RETRACING

New:

set count_running = vovset count JOBRUNNINGDURINGCRASH RETRACING+RUNNING

ISRUNNING to select Running Jobs

In the old model, counting running jobs only needed to consider the RETRACING status of a job. In the new model, you can use the ISRUNNING keyword instead.

set SELRULE "isjob status==RETRACING"

New:

set SELRULE "isjob ISRUNNING"

set subsetId [vtk_set_subselect -transient $setId "ISJOB STATUS==RETRACING ISNONEXEC==0"]

New:

set subsetId [vtk_set_subselect -transient $setId "ISJOB ISRUNNING ISNONEXEC==0"]

Is Job Started

In the old model, checking if a job started could consider just the RETRACING status. In the new model, you must check for both the RETRACING and RUNNING status.

if { $nodeInfo(status) != "RETRACING" } {
}

New:

if { $nodeInfo(status) != "RETRACING" && $nodeInfo(status) != "RUNNING" } {
}

Check that the Job is Still Running

In the old model, your code to check if a job were still running could consider only the RETRACING status. In the new model, your code should check on both the RETRACING and RUNNING status.

vtk_node_get $jobId a

if { $a(status) == "RETRACING" } {
    # job is still running
}

New:

vtk_node_get $jobId a

if { $a(status) == "RETRACING" || $a(status) == "RUNNING" } {
    # job is still running
}

Avoid Hitting the Same Job Twice

In the old model, you would need code to prevent processing the same work task twice while looking through the job nodes, because two job nodes might exist for the same work task. (A job node in RETRACING and a job node in RUNNING state can exist at the same time for the one task). In the new model, you do not need to have such code. Each work task is represented by only one job node.

foreach { id st sn jn } $JOBS {

# Prevent processing more than one job node for one work task, 
# skip nodes in RETRACING state, because the related node in RUNNING state will be the one that is processed 
 {$st == "RETRACING"} continue WACost::InitCosts? $id
}

New:

foreach { id st sn jn } $JOBS {

# every job node represents a work task
WACost::InitCosts? $id
}

Using Set Statistics to Count Running Jobs

In the old model, you could get set statistics to count running jobs by limiting the search to just job nodes with RETRACING status. In the new model, you should get set statistics to count running jobs by searching for job nodes with either RETRACING or RUNNING status.

vtk_set_statistics $setIdVovlmd setStats set stillRunning $setStats(jobs,RETRACING)

New:

vtk_set_statistics $setIdVovlmd setStats set stillRunning [expr {$setStats(jobs,RETRACING) + $setStats(jobs,RUNNING)}]