Specifying Advanced Settings

After you have created a process you can specify its priority, whether to allow parallel jobs, and whether to enable process retries.

To specify advanced settings for a process

  1. On the Navigation Tree, click Automator and select Standard  Processes.

  2. Click the name of the process you want to change or click anywhere on its row, and then click Edit button.

  3. On the displayed page, select the Advanced tab.

  4. In the Job Priority field, specify the priority for the job.
    When multiple processes are launched at the same time, the priority settings primarily determine the order they run. Other factors may affect the priority of a process. Select the priority from the following options: 

    • Lowest

    • Below Normal

    • Normal

    • Above Normal

    • Highest

  5. Select the Allow Parallel Jobs check box, if you want to allow parallel jobs.
    When parallel jobs are enabled, Automator creates a job for each of the input files assigned to a process. This setting only applies to processes that contain multiple input files when the One Project Per Job option is selected on the General tab. Enabling parallel jobs should be used in conjunction with an output naming macro that ensures the creation of unique file names throughout the duration of the process, for example, the Job Number macro.

NOTE: The order of the projects in the process will not be observed, as all jobs will execute at the same time.

  1. The Process Priority option is used for monitoring. If a number of processes have monitoring enabled and they are launched simultaneously, then processes with the highest priority run first. Any number from 1 to 255 is a valid process priority setting. Processes run in ascending numerical order according to their priority settings (for example, a process with the priority of 1 runs before a process with the priority of 2 or higher).

  1. Set the behavior of the process when exporting to .accdb, .dbf, and .mdb in the Export Behavior for accdb/dbf/mdb option. The following options are available:

    • Default - Use this option for all new processes.

NOTE: The default behavior can be defined in the config.xml file as follows:

<ServerModule Name="Datawatch.RemoteServices.DataPumpDispatcherServiceHost" MaxThreads="10">

        <AppSettings>

            <![CDATA[

<configuration>

    <appSettings>

        <add key="SdkEnableAceDaoExportByDefault" value="true" />

<!-- use "true" for ACE DAO or "false" for ACE OleDb  values-->

    </appSettings>

</configuration>

          ]]>

        </AppSettings>

</ServerModule>

  • ACE DAO - Use this option for better performance for large datasets. All exports in the system with this behavior will be executed concurrently. ACE OleDb will be used as fallback when output tables exist option is "Update existing rows" or "Update existing rows and append new.”

  • ACE OleDb - Use this option for better reliability. This behavior is slower than ACE DAO, but it allows to handle exports in parallel.

NOTE: The values for the Export Behavior for accdb/dbf/mdb option can be edited in Bulk Edits for several processes at the same time.

Take for example two jobs from different processes in a queue that exports to a different mdb.

  • If the export behavior is set to ACE DAO, then the second job will wait for the first job. The second job just wastes a free slot in the ThreadPool.

  • If the export behavior is set to ACE OleDb, then the two jobs will run in parallel.

  • In summary, two jobs in parallel will be completed in ~10 min with ACE DAO and ~20 min with ACE OleDb.

  • In case of five jobs in parallel, then they will be completed in ~25 min with ACE DAO and ~20 min with ACE OleDB.

  1. Select the Bulk Behavior for OleDb check box to enhance export performance.

NOTE: This option is applicable for Oracle Database and Microsoft SQL Server OLEDB providers only.

  1. Select the Use Transaction check box to revert the export via rollback if an error occurs during the export.

NOTE:

  • The Use Transaction check box is only available when the Bulk Behavior check box is selected.

  • Selecting this option will affect the performance of the table export and may require a large amount of system resources, especially for the target destination. Therefore, make sure to test this option first and then make the necessary adjustments.

  • The transaction is valid for the export operation only and not for the overall process.

  1. Select the Retry automatically if process prerequisites are not satisfied check box, if you want to enable process retries.

When configuring a process, one of the settings you can specify is a process retries setting. This setting allows a process to be retried if it fails for some reason, such as due to a process prerequisite not being met. When specifying a process retries setting, you may specify the interval for the retry in hours and minutes, as well as the number of retries to attempt.
The retry functionality can be useful in performing a type of polling, as well as dealing with delays in the arrival of files necessary for a process. For example, if you wish to poll for the existence of a file, you could set up a schedule that operates every 5 minutes and then set a retry that has an interval of 1 minute which retries 4 times.

  1. Specify the desired retry interval in hours or minutes in the Retry every n hours n minutes field.

  2. Select one of the following options:

  • Continue retrying for: Select this option to specify the desired retry duration in minutes, hours and/or days.

  • Continue retrying indefinitely: Select this option to continue retrying the process indefinitely until it successfully runs.

  • Retry at most n times: Select this option to enter the maximum number of times you want Automator to retry the process.

  1. Select the Once found check n times for consistent file size check box to specify that a file size check be carried out once the prerequisite is found. Also enter the number of times the check is to be carried out.
    This allows the system to cope with file inputs from slow, unreliable or multi-stage write sources. Automator will test the input file, ensuring the file size has not changed for a specified period to ensure that the file is fully ready to be processed.

  2. Specify a retry interval for checking the file size in the Retry every n minutes.

  3. Select the Retry automatically if job export failed check box, if you want to enable failed export retries. This option allows to restart the process export provided it is blocked. When specifying an export retries setting, you can specify the interval for the retry in minutes and seconds, as well as the number of retries to attempt.

  4. Specify the desired retry interval in minutes or seconds in the Retry every n minutes n seconds field.

  5. Select one of the following options:

    • Continue retrying indefinitely: Select this option to continue retrying the failed export indefinitely until it successfully runs..

    • Retry at most n times: Select this option to enter the maximum number of times you want Automator to retry the failed export.

  6. Click the Save button to save the current settings. The system saves the changes and displays the updated Process page, if the operation is completed successfully.

 

© 2024 Altair Engineering Inc. All Rights Reserved.

Intellectual Property Rights Notice | Technical Support