Starting with version 3.2, Resilio Connect supports file download prioritizing based on the filename pattern. The priority of the files that are currently being downloaded can be set in "File download priority" parameter, in Job profile.
Before creating the job, add custom parameter merge_full_availability to this job profile.
Rule syntax
Each rule starts on a new line. It contains a priority and a regex pattern that represents the files/folders in question: <priority> <regexp>
The priority level is represented with digital numbers starting with "0" (zero) which is the lowest priority - the higher the number, the higher the priority. It is allowed to have several rules with the same priority. Currently, there are no limits to the number of rules and/or priorities. But the job's performance degrades as the complexity of the regexp grows.
The regex body is recorded between starting and ending slash (/) as seen on the example. Backslash (\) is used to escape special characters, flag /i used to make the rule case insensitive.
If the syntax of a rule is incorrect, the Agent will report an error.
The example of the rules are:
9 /\.ptx$/
8 /\.prproj$/
3 /^test/
2 /.+copy.*/
1 /file[1-3]\.txt$/
How files are evaluated for priority
The order of the rules is important. Each file/folder that needs to be synced is checked against the rule line by line till a first matching rule is discovered, this line's priority level is assigned to the file/folder. This file/folder is no longer checked against other rules.
All files that don't match any of the rules are downloaded with lowest priority (0, zero). It's not obligatory to use a 0-priority rule in the configuration, unless you explicitly want to prioritize specific files as lowest priority.
It all does not necessarily mean that rules shall be sorted from highest to lowest priority. To illustrate the idea, see the below:
1 /directory1\abc1.txt/ 2 /directory1*/
File abc1.txt inside directory1 matches the first rule and will be downloaded with low priority. Other files in directory1 with be synced with higher priority. Compare this behavior to a similar rule, but different order
2 /directory1*/ 1 /directory1\abc1.txt/
All files in directory1 match the first rule and all will be synced with high priority, even abc1.txt. Second rule won't be even checked for these files.
How the download priority works
Resilio Connect Agents can prioritize only downloads, not uploads. Only active file queue can be prioritized, which is 100,000 files currently.
Resilio Agent can notice that a file with higher priority appeared in the queue so it will pause current transfers of lower priority, and resume them later when higher-priority transfers are completed. There are some limitations in this behavior, see below.
Resilio Agent does start the download according to the priority of a file. However, it does not guarantee that the download will be completed in the same order, as multiple downloads are being done in parallel. Files of the same priority are not prioritized against each other.
Please note that the files seen on Job run -> Agents -> {agent_name} -> Files tab do not show the actual priority/order of download! It simply shows files' download status and its timestamp.
Limitations and peculiarities
- Resilio Agent only prioritizes the start of the downloading. A low-priority file download may begin being downloaded while a high-priority file is still in progress. It may also even finish download sooner, depending on multiple factors, for example, file size.
- Small files, synced without being split into pieces (with direct torrent), are not paused and continue being synced even if a more high-priority files appear. It saves performance, not having to switch the context.
- With a too low bandwidth speed limits (1 Mbps), low priority files may fail to be paused if a more high-priority file appears.
- Clearing file download priority in the job profile is not supported (the job gives an error about invalid rule). The job needs to be recreated.
- Writing comments in the priority rule is supported and shall start with # on a new line. Empty lines are supported.