• Contents
  • Index

Altair® Panopticon™

 

Connector for Kafka Publisher

The Kafka Publisher connector allows a tuple to be published to a Kafka topic.

Steps:

 

1.     Enter the following properties:

Property

Description

Bootstrap Server

List of host/port pairs of Kafka servers used to bootstrap connections to a Kafka cluster.

By default, the value is localhost:9092. However, this can be overridden by specifying another bootstrap server in the External Settings text box.

Schema Registry Host

Where the Schema Registry is located. This can be in a different location from the Kafka cluster.

Schema Registry Port

The port number of the schema registry which provides the serving layer for the metadata. Default is 8081.

 

2.     To support authentication (i.e., username and password), enter the system settings in the External Settings box.

 

   NOTE

If the bootstrap server is not secure, then there is no need to authenticate, and you may leave the External Settings blank.

 

Below is an example of system settings for an SASL authentication:

bootstrap.servers=localhost:9093 sasl.jaas.config= org.apache.kafka.common.security.plain.PlainLoginModule required username="dwchuser" password="dwchpwd";

 

3.   Click  to populate the drop-down list and select a Topic.

 

   NOTE

·           Ensure that the ability to ping is enabled in the ZooKeeper Host. Otherwise, if ping is disabled, the Fetch Topics button will not be able to populate the list of topics and you need to manually enter the topic names.

·           For Avro format messages, make sure to select an output topic. This populates the list of columns, with the data type found from inspecting the first ‘n’ rows of the file.

 

 

4.   For non-Avro format messages, select Json in the Message Composer drop-down list box.

5.   Check the Use Schema Registry box to support Avro and JSON serialization formats.

6.   Enter the Timeout or the length of time to wait for the server response. Default is 5 (in seconds).

7.   Click  to add columns to the Kafka connection that represent sections of the message.

8. Then enter or select:

·         Name

·         Type (Numeric, Text, or Date/Time)

·         Value (can either be a parameter or data entry that can be used as a publish value)

To delete a column, check its  or all the column entries, check the topmost , then click .