Search

GlobalSCAPE Knowledge Base

Advanced Workflows affect system performance

Karla Marsh
EFT

THE INFORMATION IN THIS ARTICLE APPLIES TO:

  • EFT Server Enterprise version 6.2 and later
  • EFT pre-v8.x stores advanced properties in the registry.
  • EFT v8.x and later store Advanced Properties in a JSON file. When you upgrade from EFT v7.4.x to EFT v8, the non-default settings that you have defined in the registry will be added to the Advanced Properties file during upgrade. (Default settings are part of the EFT configuration files.) For a spreadsheet of advanced properties, please refer to the "Advance Properties" topic in the help for your version of EFT.
  • See also:
  • Using Advanced Properties to optimize running asynchronous events

    How can I limit the number of event rule subroutines running?

SYMPTOM

Multiple, simultaneous Advanced Workflow tasks affect system performance.

RESOLUTION

Adjust the maximum number of simultaneous tasks that are allowed to run and the number of simultaneous Advanced Workflow tasks that can process the tasks.

Please contact GlobalSCAPE Technical Support for guidance before adjusting these settings.

The following values affect the processing of Advanced Workflow tasks:

  • RunningTaskLimit is used when running sync tasks to control the total number of Advanced Workflow tasks that can execute simultaneously (sync).
  • GSAWEMaxiumQueuedTasks is used for ASYNC tasks to control the maximum number of Advanced Workflow tasks that can be queued before dropping new tasks. 0 = no limit (the default). If you set it for 20, for example, and there are 20 tasks queued, no other tasks will be run until the queue is available.

To set the properties:

  • Prior to v8.x, create the values in the registry: DWORD: HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\GlobalSCAPE Inc.\EFT Server 4.0\AWE
  • In v8.x and later, add the values to the advancedproperties.json file.
  • {
    "RunningTaskLimit":20
    "GSAWEMaxiumQueuedTasks":20
    }

- Given the default values, only 3 tasks will run in a single instance of the Advanced Workflow instances at a time, even if more are called for.
- For these changes to take effect you must stop the EFT server service, make the requisite registry change, and then restart EFT server service.
- You may see fewer than the expected number of GSAW engine instances using process monitor or similar tools while testing. This is likely due to Folder Monitor threads which grow and shrink dynamically.
- Which value should you choose? It largely depends on what you are trying to accomplish and system resources available. Setting values of 50:50 (or 100:100) is highly efficient in terms of executing multiple tasks simultaneously with minimal impact to system resources, but there is an increased risk of failure due to the various task threads exiting completed tasks when other tasks are still being processed or haven’t yet started.

Examples of Using RunningTaskLimit

Example 1:

Using the following settings:

  • AWE Concurrent Task = 3 
  • AWE Queue = 0
  • EFT AP RunningTaskLimit = 1
  1. A Globalscape Test Engineer triggered 10 AWE tasks and a total of 10 tasks executed one at a time.
  2. One task max ran at a time, since RunningTaskLimit= 1, and the rest were put into EFT's queue, so only one task at a time was sent to AW, staying under its limit of 3, meaning no tasks were put in the AW queue.

Example 2:

Using the following settings:

  • AW Concurrent Task = 1 
  • AW Queue = 0 
  • EFT AP RunningTaskLimit = 3

 A Globalscape Test Engineer triggered 10 AW tasks and a total of 8 tasks that executed one at a time.

  1. EFT limits came into play, sending 3 AW tasks over to AWs API.
  2. AW then executed 1 of the 3 and failed the other 2, since the queue is set to 0.
  3. EFT then sent over 1 task at a time until 7 other AWs in its queue are executed, 1 at a time.

They key takeaway is that the AW Concurrent Task should be a bit higher than EFTs AP RunningTaskLimit to avoid AW tasks being queued on the AW side as this can lead to tasks being dropped.

Also, tasks queued on the AW side do not show up when looking at EFTs Perfmon counters.
We also noticed that when AWs are dropped on the AW side, EFT does not properly remove them from its actively running AW task queue, causing other issues when more AWs are run later.

The table below shows various outcomes assuming a Folder Monitor operation in which 100 files are deposited:

Running Task Limit

Number of AWE Engine instances

System Affect (mem, CPU)

Efficiency

Risk of failure due to threading

Explanation and Pros/Cons.

3 (default prior to v7)

1

Very low

Low

Low

A single Advanced Workflow service instance processes 3 tasks at a time. Low impact on resource but takes longer to process all 100 tasks.

In EFT v7 and later, the default is 10
10 Med
Med Zero EFT will run 10 Advanced Workflow tasks concurrently, each in their own GSAWE.exe engine

20

1

Low-

med

Medium

Medium

A single Advanced Workflow instances processes 20 tasks at a time. Moderate impact but processes faster. Slightly increased risk of failure due to threading issues.

40

2

Med

Med-

high

Med-

high

Two Advanced Workflow instances processes 40 tasks concurrently (20 each). Higher impact and 2X memory utilization for GSAWE (engine) process.

100

1

Med

High

High

Single Advanced Workflow instance processes all 100 tasks concurrently. Only moderate performance hit but highly efficient processing. Downside is increased risk of failure due to Advanced Workflow threading issues.

100

5

Med-

high

High

Med-

high

Five Advanced Workflow instances processes 20 tasks each concurrently. Increased impact to resources, great performance, low-moderate risk of threading issues.

20

20

High

Med

Zero

Twenty Advanced Workflow instances processes a single task each. Significant impact to resources and memory utilization, moderate performance, zero risk of threading issues.

100

100

Very

High

Very

high

Zero

Eliminates threading related problems by executing one Advanced Workflow instance for each task. Problem is memory consumption for the up to 100 Advanced Workflow instances. RAM upgrade may be in order.

Details
Last Modified: 2 Months Ago
Last Modified By: kmarsh
Type: FIX
Rated 2 stars based on 16 votes.
Article has been viewed 73K times.
Options
Also In This Category
Tags