Advanced Workflows affect system performance

THE INFORMATION IN THIS ARTICLE APPLIES TO:

• EFT Server Enterprise version 6.2 and later
• EFT pre-v8.x stores advanced properties in the registry.
• EFT v8.x and later store Advanced Properties in a JSON file. When you upgrade from EFT v7.4.x to EFT v8, the non-default settings that you have defined in the registry will be added to the Advanced Properties file during upgrade. (Default settings are part of the EFT configuration files.) For a spreadsheet of advanced properties, please refer to the "Advance Properties" topic in the help for your version of EFT.
• See also:

Using Advanced Properties to optimize running asynchronous events

How can I limit the number of event rule subroutines running?

SYMPTOM

Multiple, simultaneous Advanced Workflow tasks affect system performance.

RESOLUTION

Adjust the maximum number of simultaneous tasks that are allowed to run and the number of simultaneous Advanced Workflow tasks that can process the tasks.

Please contact GlobalSCAPE Technical Support for guidance before adjusting these settings.

The following values affect the processing of Advanced Workflow tasks:

• RunningTaskLimit is used when running sync tasks to control the total number of Advanced Workflow tasks that can execute simultaneously (sync).
• GSAWEMaxiumQueuedTasks is used for ASYNC tasks to control the maximum number of Advanced Workflow tasks that can be queued before dropping new tasks. 0 = no limit (the default). If you set it for 20, for example, and there are 20 tasks queued, no other tasks will be run until the queue is available.

To set the properties:

• Prior to v8.x, create the values in the registry: DWORD: HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\GlobalSCAPE Inc.\EFT Server 4.0\AWE
• In v8.x and later, add the values to the advancedproperties.json file.

{
"RunningTaskLimit":20
"GSAWEMaxiumQueuedTasks":20
}

- Given the default values, only 3 tasks will run in a single instance of the Advanced Workflow instances at a time, even if more are called for.
- For these changes to take effect you must stop the EFT server service, make the requisite registry change, and then restart EFT server service.
- You may see fewer than the expected number of GSAW engine instances using process monitor or similar tools while testing. This is likely due to Folder Monitor threads which grow and shrink dynamically.
- Which value should you choose? It largely depends on what you are trying to accomplish and system resources available. Setting values of 50:50 (or 100:100) is highly efficient in terms of executing multiple tasks simultaneously with minimal impact to system resources, but there is an increased risk of failure due to the various task threads exiting completed tasks when other tasks are still being processed or haven’t yet started.

Examples of Using RunningTaskLimit

Example 1:

Using the following settings:

• AWE Concurrent Task = 3 
• AWE Queue = 0
• EFT AP RunningTaskLimit = 1

1. A Globalscape Test Engineer triggered 10 AWE tasks and a total of 10 tasks executed one at a time.
2. One task max ran at a time, since RunningTaskLimit= 1, and the rest were put into EFT's queue, so only one task at a time was sent to AW, staying under its limit of 3, meaning no tasks were put in the AW queue.

Example 2:

Using the following settings:

• AW Concurrent Task = 1 
• AW Queue = 0 
• EFT AP RunningTaskLimit = 3

 A Globalscape Test Engineer triggered 10 AW tasks and a total of 8 tasks that executed one at a time.

1. EFT limits came into play, sending 3 AW tasks over to AWs API.
2. AW then executed 1 of the 3 and failed the other 2, since the queue is set to 0.
3. EFT then sent over 1 task at a time until 7 other AWs in its queue are executed, 1 at a time.

They key takeaway is that the AW Concurrent Task should be a bit higher than EFTs AP RunningTaskLimit to avoid AW tasks being queued on the AW side as this can lead to tasks being dropped.

Also, tasks queued on the AW side do not show up when looking at EFTs Perfmon counters.
We also noticed that when AWs are dropped on the AW side, EFT does not properly remove them from its actively running AW task queue, causing other issues when more AWs are run later.

The table below shows various outcomes assuming a Folder Monitor operation in which 100 files are deposited: