Sets

Sets of Nodes

The nodes in the dependency graph can be grouped into named sets. The names of the sets are unique. It is not allowed to have two sets with the same name.

Sets have many uses in VOV:
  • They are used by the server to maintain the internal organization of the graph
  • They are used to focus the attention on the problem of interest
  • They are used to generate reports
  • They are used to define retrace targets.

Set Status

The status of a set is computed from the status of the nodes it contains.

The status of a set is computed from the Node Status of the nodes it contains. If the set is empty, it's status is EMPTY. Otherwise, the status is determined by the dominant status according to the following ranking:
  1. RUNNING
  2. RETRACING
  3. FAILED
  4. INVALID
  5. UNKNOWN
  6. MISSING
  7. DELETED
  8. SLEEPING

If at least one node is RUNNING, then the set itself is RUNNING. Otherwise, if at least one node is RETRACING, then the whole set is RETRACING and so on.

In particular, if the set status is VALID (that is, green), then the set has no INVALID, FAILED, RETRACING or RUNNING nodes.

The utility vovset can be used to create, modify, recompute, destroy sets.

vovset

Perform basic operations on sets of nodes.

Note: nc modify -res now support binary unit conversion for all memory based resources as a convenience from Petabytes (PB), TerabyteS (TB), or GigabyteS (GB) to Megabytes (MB), which is still used internally and reported by all commands. The input conversion will accept either decimal or integer form and are all case-insensitive, so for example both nc run -r SWAP/1GB — sleep 0, and nc run -r RAM/0.1Tb — sleep 0 are supported.

The currently supported parameter names for which this conversion is supported are RAM/, RAM#, RAMFREE#, RAMFREE/, RAMTOTAL#, RAMFREE/, SWAP/, SWAP#, SWAPFREE#, SWAPFREE/, SWAPTOTAL#, SWAPTOTAL/ and TMP# or TMP/. By default the unit is MB (Megabytes), where 1MB is 1<<20 bytes.


vovset: Usage Message
DESCRIPTION: 1
    Perform basic operations on sets of nodes.

USAGE:
    % vovset attach    <setName> <nodeId> ...
    % vovset clear     <setNameOrId>
    % vovset count     <setNameOrId> [<status>|<status>+<status>|ALL] ...
    % vovset create    [-smart|-smartc] [-universe <setName>]  <NewName>
                       <Rule>
    % vovset detach    <setName> <nodeId> ...
    % vovset exists    <setName>
    % vovset geo       <setNameOrId> [-v] [-h] [-f <file>] [-forget] [-raw]
    % vovset info      <setNameOrId> ...
    % vovset list      [-rx <filter>] [-all|-names|-ids]
    % vovset memberof  <Id> ...
    % vovset forget    <setNameOrId> ...
    % vovset recompute <setNameOrId> ...
    % vovset rename    <setNameOrId> <newName>
    % vovset resources <setNameOrId> <newResourceList>
    % vovset retrace   <setNameOrId> ...
    % vovset show      [-rx <filter>] [-sort] [-O <format>]
                       <setNameOrIdOrRule> ...
    % vovset stats     <setNameOrId> ...
    % vovset tat       <setName> [-O raw]
    % vovset union     [-clear|-keep|-append] <largeSetResult> <smallset1> ...
    % vovset upcone    <srcSetName> [dstSetName]
    % vovset downcone  <srcSetName> [dstSetName]
    % vovset zippable  <setNameOrId> <0|1>

ACTIONS:
    attach           -- Attach elements (nodes or subsets) to a set.
                        Create the set if necessary.
    clear            -- Detach all nodes from set.
                        New set is created (empty) if necessary.
    count            -- Count nodes of a given status in the set.
    create           -- Create a new set with given name and rule.
    detach           -- Detach elements from a set.
    exists           -- Returns 0 or 1 depending on existence
                        of set specified by name.
    geo              -- Get or set geometry information for elements
                        of the set.
    info             -- Get basic info about a set.
    list             -- List all sets.
    forget           -- Forget given sets.
    memberof         -- Show IDs and names of sets to which each ID belong
    invalidate       -- Invalidate the given sets.
    recompute        -- Recompute the given sets based on their
                        selection rule.
    rename           -- Rename a set.
    resources        -- Change resources of jobs in set.
    retrace          -- Retrace given sets with
                        NORMAL priority in SAFE mode.
                        Use vsr to change mode and/or priority.
    show             -- Show elements in a set given by name, by id,
                        or by selection rule.
    stats            -- Get statistics about a set.
    tat              -- Get Turn-Around-Time and other stats
    timereport       -- Undocumented action -- do not use.
    union            -- Create a set from the union of zero or
                        more sets to a set.
                        If option -clear is used, the result set is 
                        cleared before the union operation (default)
                        If option -keep is used, the result set is kept.

    upcone           -- Get IDs of the upcone of the set members.  When
                        destSetName is given, upcone elements become members.
    downcone         -- Like upcone, but gets the downcone of set members.
    zippable         -- Set or reset the zippable flag in all
                        files in the set.

    Action verbs are case insensitive and can be abbreviated
    to the shortest unique sequence, except "forget" "create"
    and "recompute" which must be spelled out completely.

OPTIONS:
    -rx   <regexp>   -- Regular expression to filter output.
    -rule <selrule>  -- Apply a selection rule in 'show'.
    -O    <format>   -- Format of output for 'show'.
    -sort            -- Sort output for 'show'.

EXAMPLES:
    % vovset attach "MySet" 2345 2468
    % vovset clear "MySet"
    % vovset count "MySet"
    % vovset count "MySet" VALID
    % vovset count "MySet" VALID RUNNING
    % vovset count "MySet" VALID+FAILED  SCHEDULED
    % vovset count "MySet" ALL VALID RUNNING
    % vovset create "MyEmptySet" ""
    % vovset create "ToDo" "isjob status==INVALID"
    % vovset create -smart "AllFailed" "isjob status==FAILED"
    % vovset forget some_set
    % vovset geo    "MySet"
    % vovset geo    "MySet" -raw
    % vovset geo    "MySet" -f file.geo
    % vovset list
    % vovset list  -rx "UPCONE"
    % vovset retrace "ToDo"
    % vovset show  -O '@ID@ @DURATIONPP@' 'System:jobs'
    % vovset show  'isjob status==FAILED'
    % vovset tat  System:jobs
    % vovset tat  -O raw System:jobs
    % vovset union OurSet YourSet MySet
    % vovset union -clear OurSet YourSet MySet
    % vovset union -keep OurSet YourSet MySet
    % vovset zippable "Some_set" 1