THE INFORMATION IN THIS ARTICLE APPLIES TO:
QUESTION
Is there a thread limit as to how many files can be transferred via the same Event Rule?
ANSWER
The Folder Monitor Action threads use a dynamically growing pool. The system will keep track of the number of active threads over time and periodically calculate the average of number of concurrent active threads over that time period. The sample rate is once every 5 seconds, and the sample period is 10 samples. After sampling 10 times and finding the average concurrent active threads over that period, the system determines if the past average is 85% of the current active thread count or higher and, if YES, then grows that pool by 1/2 of the concurrent active threads, up to a set maximum number of threads.
Therefore, if we are currently running close to or above the prior average of concurrent threads, we should grow our thread pool to allow for room for any more Events that are coming in to be processed. By default, we start with 3 threads in the pool per site, and can grow to a maximum of 64 threads.
The advance property below will override the initial worker thread count. If the value is bigger than the default 32 maximum thread pool size, the registry value will also become the new maximum pool size. So, if the registry value is greater than 64, then we will have a static pool of threads of that designated size (because the initial and maximum values are both the same).
Maximum Concurrent Transfer Threads During Upload or Download
A new instance of an Event Rule is created each time you click Run Now. During upload and download operations, EFT Server's transfer engine compares the "Max concurrent transfer threads" value specified in the Advanced Options dialog box with the number of current connections to the host. The transfer engine object is unique for each Event Rule and cannot exceed the allowed number of connections to the host; that is, each Event Rule will wait until the number of connections to the host falls below the level of "Max concurrent transfer threads" specified in the Advanced Options dialog box. The Rules will be processed in the order of the download/upload queue.
EFT v8.0 and later store Advanced Properties in a JSON file. When you upgrade from EFT v7.4.x to EFT v8, the non-default settings that you have defined in the registry will be added to the Advanced Properties file during upgrade. (Default settings become part of the EFT configuration files.) For a more on how to use advanced properties, and a spreadsheet of the advanced properties, please refer to the "Advanced Properties" topic in the help for your version of EFT.
In EFT v8 and later:
Add the name:value pair to the AdvancedProperties.JSON file in EFT's \ProgramData\ directory.
{
"FolderMonitorWorkerThreadCount": 64
}
To override the default:
The logic behind Event Rules execution and “Max concurrent transfer threads” setting is the following:
-
Each Event Rule (except Folder Monitors) is executed in the thread in which the corresponding Event arises. Folder Monitor Rules are executed in one of the threads from a folder monitor processing thread pool (folder monitor thread pool size limit can affect this). If you press “Run Now,” a separate thread for the Event execution is created.
-
Each Event Rule executes its Actions sequentially.
-
Copy/Move/Download Action implementation does some processing (e.g., parsing Event variable, etc.) and passes transfer parameters (local path, remote path, protocol, advanced options, etc.) to the Transfer Engine, then waits for transfer to complete.
-
The Transfer Engine turns the transfer task into one or more transfer items (e.g., in the case where one actual file is in the task, more in the case of wildcard upload/download, each of the items specify actual local and remote path) and puts them into the queue for further processing. Each of transfer items has its own “Max concurrent transfer threads” value inherited from transfer task parameters.
-
The Transfer Engine has its own thread pool for processing transfer items. If the thread from the pool is considered active, then it’s actually doing some work for some transfer items (connecting, transferring, navigating or some local processing stuff).
- The transfer item is not taken into processing when the number of active threads is greater than its “Max concurrent transfer threads” setting value.
For EFT versions prior to v8, add the advanced property (DWORD) to the registry:
32-bit:
HKEY_LOCAL_MACHINE\SOFTWARE\Globalscape Inc.\EFT Server 4.0\EventRules\FolderMonitorWorkerThreadCount
64-bit:
HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Globalscape Inc.\EFT Server 4.0\EventRules\FolderMonitorWorkerThreadCount