Search

GlobalSCAPE Knowledge Base

Changing the Number of Concurrent Threads Used by Event Rules

Karla Marsh
EFT Express (SMB) & Enterprise

THE INFORMATION IN THIS ARTICLE APPLIES TO:

  • EFT Server, all versions

QUESTION

Is there a thread limit as to how many files can be transferred via the same Event Rule?

ANSWER

The Event Rule Monitor Folder process is limited to 3 concurrent threads by default. This means that if you have 5 Folder Monitor Event Rules monitoring the same folder, and a file is added to the monitored folder, only 3 of the 5 Rules will fire, as determined by the operating system. The 4th and then 5th Rule execute only when one or more of the first three threads are done firing and executing any Actions. If you have, for example, 100 concurrent Monitor Folder Event Rules, they are not all triggered simultaneously.

To override the default, create the following key:

32-bit:

HKEY_LOCAL_MACHINE\SOFTWARE\Globalscape Inc.\EFT Server 4.0\EventRules\FolderMonitorWorkerThreadCount

64-bit:

HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Globalscape Inc.\EFT Server 4.0\EventRules\FolderMonitorWorkerThreadCount

DWORD: the number of allowed concurrent threads

As of version 6.1, the Folder Monitor Action threads use a dynamically growing pool. The system will keep track of the number of active threads over time and periodically calculate the average of number of concurrent active threads over that time period. The sample rate is once every 5 seconds, and the sample period is 10 samples. After sampling 10 times and finding the average concurrent active threads over that period, the system determines if the past average is 85% of the current active thread count or higher and, if YES, then grows that pool by 1/2 of the concurrent active threads, up to a set maximum number of threads.

Therefore, if we are currently running close to or above the prior average of concurrent threads, we should grow our thread pool to allow for room for any more Events that are coming in to be processed. By default, we start with 3 threads in the pool per site, and can grow to a maximum of 32 threads.

The above Registry Entry will override the initial worker thread count. If the value is bigger than the default 32 maximum thread pool size, the registry value will also become the new maximum pool size. So, if the registry value is greater than 32, then we will have a static pool of threads of that designated size (because the initial and maximum values are both the same).

Maximum Concurrent Transfer Threads During Upload or Download

A new instance of an Event Rule is created each time you click Run Now. During upload and download operations, EFT Server's transfer engine compares the "Max concurrent transfer threads" value specified in the Advanced Options dialog box with the number of current connections to the host. The transfer engine object is unique for each Event Rule and cannot exceed the allowed number of connections to the host; that is, each Event Rule will wait until the number of connections to the host falls below the level of "Max concurrent transfer threads" specified in the Advanced Options dialog box. The Rules will be processed in the order of the download/upload queue.

The registry DWORD value "MaxNumberConnections" determines the global maximum number of connections for EFT Server's transfer engine. (The default value is 100).

32-bit:

HKEY_LOCAL_MACHINE\SOFTWARE\Globalscape\TED 6\Settings\Connection

64-bit:

HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Globalscape\TED 6\Settings\Connection

DWORD value "MaxNumberConnections"

The logic behind Event Rules execution and “Max concurrent transfer threads” setting is the following:

  1. Each Event Rule (except Folder Monitors) is executed in the thread in which the corresponding Event arises. Folder Monitor Rules are executed in one of the threads from a folder monitor processing thread pool (folder monitor thread pool size limit can affect this). If you press “Run Now,” a separate thread for the Event execution is created.

  2. Each Event Rule executes its Actions sequentially.

  3. Copy/Move/Download Action implementation does some processing (e.g., parsing Event variable, etc.) and passes transfer parameters (local path, remote path, protocol, advanced options, etc.) to the Transfer Engine, then waits for transfer to complete.

  4. The Transfer Engine turns the transfer task into one or more transfer items (e.g., in the case where one actual file is in the task, more in the case of wildcard upload/download, each of the items specify actual local and remote path) and puts them into the queue for further processing. Each of transfer items has its own “Max concurrent transfer threads” value inherited from transfer task parameters.

  5. The Transfer Engine has its own thread pool for processing transfer items. If the thread from the pool is considered active, then it’s actually doing some work for some transfer items (connecting, transferring, navigating or some local processing stuff).

  6. The transfer item is not taken into processing when the number of active threads is greater than its “Max concurrent transfer threads” setting value.

Details
Last Modified: 9 Months Ago
Last Modified By: kmarsh
Type: HOWTO
Article not rated yet.
Article has been viewed 33K times.
Options
Also In This Category
Tags