Advanced REST Usage

This section will cover some more advanced REST API topics: management of JWT access tokens, HTTPS secure and insecure connections, and connection keep-alive. The example application, nc_info.py, imitates the familiar Accelerator CLI command nc info, which returns information about a job. This application will interface to the REST API directly, instead of using the vov_rest_v3 module interface layer.

This REST application requires the NC_URL environment variable to be set to the Accelerator queue URL, as returned by nc cmd vovbrowser. Also, this Python program needs JWT-handling module getToken.py from the next section in this guide. Create that python file before running nc_info.py.

Here is a shell session that shows how to run nc_info.py. In this example, two jobs are started, and an invocation of nc_info.py will query and display information about the two running NC jobs.

% nc run -v 2 sleep 123
Job <000001138> is submitted
% nc run -v 2 sleep 456
Job <000001143> is submitted
% export NC_URL=`nc cmd vovbrowser`
% echo I will need `ls getToken.py`
I will need getToken.py
% ./nc_info.py 000001138 000001143
Password:
Job             000001138
User,Group      user99,/time/users.user99
Command         vw sleep 123 > vnc_logs/20211012/132628.122875
Status          Running
   Host         myhost
   Duration     31s

Job             000001143
User,Group      user99,/time/users.user99
Command         vw sleep 456 > vnc_logs/20211012/132633.122899
Status          Running
   Host         myhost
   Duration     26s

nc_info.py

#!/usr/bin/python
#
# nc_info.py
#
# Usage
#
#      export NC_URL=<URL FOR NC QUEUE>
#      ./nc_info.py JOB_ID [JOB_ID ...]
#

import os, sys, json, requests
from getToken import getJWT

def getMyPassword():
    import getpass
    return getpass.getpass('Password:')

def infoPrint( text ):
    dd =  json.loads(text)
    id    = find(dd, 'ID')
    user  = find(dd, 'USER')
    group = find(dd, 'GROUP')

    print ("%-16s%s" % ("Job", id) )
    print ("%-16s%s,%s" % ("User,Group", user, group))
    print ("%-16s%s" % ("Command", find(dd, 'COMMAND')))
    print ("%-16s%s" % ("Status", find(dd, 'STATUSNC')))
    print ("%-16s%s" % ("   Host", find(dd, 'HOST')))
    print ("%-16s%s" % ("   Duration", find(dd, 'DURATIONPP')))

def find(jobdump, key):
    for c in range (0, len(jobdump['columns'])):
        if (jobdump['columns'][c]['field'] == key):
            break
    return jobdump['rows'][0][c]

#
# Main body
#
nc_url = os.environ['NC_URL']
scheme = nc_url.split(":")[0]
hostport = nc_url.split("/")[2]
url = "{0}://{1}".format(scheme, hostport)

ss = requests.Session()   # use keep-alive
jwt = getJWT(ss, url, os.environ["USER"], getMyPassword())
for arg in range (1,len(sys.argv)):
    jobid = sys.argv[arg]
    query = url + '/api/v3/jobs/' + jobid
    r = ss.get(query, headers={"Authorization": jwt},
		 verify=True)
    infoPrint(r.text)
    print ("")

JWT Access Tokens

To authorize REST access via the API, REST requests must pass an access token in the request header. In the nc_info.py example, the jwt variable holds the access token. Access tokens expire about 4 hours after issue. An application that will run for several hours should adopt some strategy to allow for access token expiry. One strategy would be to re-authenticate, using login name and password, prior to any burst of REST activity that will be known to be complete in a few hours or less. Another strategy is to check the request return status and re-authenticate at that time, using the new access token thereafter.

The recommended way to authenticate user name and password to issue an access token is the VOVRestV3 Python class authorize() method function. If you would like direct control over the access token allocation, as in the nc_info.py example, see the getJWT() function in the getToken.py example in the next section.

HTTPS Security Considerations

REST requests and responses are sent over the HTTP communication protocol, and suitable HTTP connections can be configured with three possible security levels, ranging from insecure to very secure:
  1. The basic (insecure) way to start the Accelerator queue server and web server is to use the default HTTP connection on the server's webport. The REST API interface always is allowed on HTTP webport connections.
  2. An intermediate level of security is possible if the HTTPS protocol is requested by configuring server parameter ssl.enable to 1 in policy.tcl. In this case, a self-signed SSL certificate will be generated when the Accelerator server starts. If the REST application uses the VOVRestV3 python module method functions, this type of connection will be supported without warnings being issued. If direct access to the REST API is implemented using the Python request method functions get() and post(), then communication will be allowed on these connections only if the optional verify=False keyword argument is passed to those functions.
  3. A very secure HTTPS connection will be established if an SSL certificate that was signed by a trusted Certificate Authority (CA) is added to the Accelerator server configuration. This is the recommended way to configure Accelerator products. If the REST application author would like to require level 3 security, then the direct access to REST via request method functions get() and post() must be used, and the keyword argument verify=True must be specified.

Connection Keep-Alive

The use of HTTP keep-alive, or persistent connection, is an important technique that optimizes applications during times of very frequent REST requests. HTTP keep-alive and the resultant reuse of HTTP connections will occur when using the VOVRestV3 submitRequest() method function or when using a "session" allocated by the request.Session() method function.

The nc_info.py Python application illustrates the use of keep-alive by the latter method in its main loop across job IDs. Each call to ss.get() will reuse the same HTTP connection. If the ss.get() call were to be replaced by request.get(), the keep-alive feature would not persiste an HTTP connection between subsequent requests. In that case, each request would build a new HTTP connection before issuing the request.