Selecting and Defining the Data Connector File Source
Several connectors including JSON, Text, XML, and Stream Simulator, allow selection from a File, Web URL, or Text source.
Select the connector file source:
- File
You can either:
- Upload a data source snapshot by clicking Upload File
then Browse
to browse to the file source.
After selecting the file, it is displayed with the timestamp of the snapshot.
To change the data source, click
then Browse
to browse to a new file.
- Link to a data source file by clicking Link to File
and entering a File Path.
Ensure that in a cluster, you need to use a a shared path, or put it on every node and use a path that resolves on every node. You can update its contents whenever you want.
NOTE: An error notification displays upon fetching sheets/column generation when the File Path is blank.
Click Close and enter the file path.
- Upload a data source snapshot by clicking Upload File
- Text
Then enter the text block to be parsed.
NOTE:
- The Text file source is not available for the MS Excel connectors.
- An error notification displays upon column generation when the Text box is blank.
Click Close and enter the text block.
- Web URL
The dialog changes to allow specification of the following:
Property Description Authentication Type
-
Basic
The basic authentication.
-
OAuth
Then enter the following settings:
-
Token URL – The URL to retrieve the access token from.
-
Token Request Body – The request body used for access token requests.
-
Add Access Token To - The Access token retrieved from the Token URL can be added to headers, URL or request body, depending on how the endpoint needs the token.
-
Request Headers - A header is automatically added to the REST API request.
-
Request URL - The URL needs to be manually parameterised with a {access_token} parameter, before calling the REST API, the parameter is replaced with the actual token.
-
Request Body - The Request Body needs to be manually parameterised with a {access_token} parameter, before calling the REST API, the parameter is replaced with the actual token.
-
-
NOTE:
-
The given request body is posted to the Token URL as application/x-www-form-urlencoded, so request body must be formatted like field1=value1&field2=value2, e.g., client_id=xxxx&client_secret=xxxx&grant_type=client_credentials.
-
Not available in the Stream Simulator connector.
Path
The absolute path including the HTTP where the file is located.
Proxy Server URI
The HTTP Proxy setting that will allow the connector to reach the endpoint.
Headers
-
Headers are separated by a comma
-
Each Header is entered as Name = Value, where Name and Value can be enclosed in double quotes to allow inclusion of any character except for double quotes
-
Name and Value can also be left unquoted, in which case they may not include comma or equals characters
Content Encoding
Select the Content Encoding with the HTTP Header: None, GZip, Deflate, or GZip and Deflate
User Id
The user Id that will be used to connect to the connector’s service.
Password
The password to connect to the connector’s service.
HTTP Method
Select the appropriate HTTP method for the request from the following options:
-
GET – Retrieve data
-
POST – Add new data
-
PUT – Replace existing data
-
DELETE – Remove existing data
Timeout
The length of time to wait for the server response (10 to 300). Default is 10.
Request Body
The Request Body for the HTTP POST.
Content Type
The required Content Type. Default is application/x-www-form-urlencoded
Record Path
The record path that will be queried by the connector’s path (e.g., myroot.items.item).
File Encoding
Set the character encoding to use in text data.
-
UTF-8
-
UTF-16
-
UTF-32
-
US-ASCII
-
Windows-1252
NOTE: An error notification displays upon column generation when the Path is blank.
Click Close and enter the web path.
-
(c) 2013-2024 Altair Engineering Inc. All Rights Reserved.