Altair® Panopticon

 

Connector for Livy Spark

Livy is an open-source REST interface for interacting with Apache Stark. It supports executing snippets of code or programs such as Scala, Python, Java, and R in a Spark context that runs locally or in Apache Hadoop YARN.

The Livy Spark connector allows you to run these codes and fetch the data in Panopticon Real Time.

Steps:

 

1.     Enter or select the following properties:

Property

Description

Host

Livy Spark host address.

User Id

User Id that will be used to connect to Livy Spark.

Password

Password that will be used to connect to Livy Spark.

Kind

Currently, the supported kind of connection to be used is pyspark (Interactive Python Spark session).

Request Timeout

Length of time to wait for the server response. Default is 30.

Polling Count

The number of polling done to the Livy Spark server to check if the status of the app is successful. Default limit is 150.

Polling Frequency (in seconds)

Frequency of the polling. Default is 2.

Script

The script to use.