RAM optimized job

Resilio Connect Agents v3.0.0 greatly reduced consumption of RAM, by around 85%, when keeping hundreds of millions of files in sync compared to previous versions. The Agents require around 256 bytes of RAM per file or subfolder, compared to 2-4 Kb in older versions.

This was made possible by changing the way the Agents keep in sync with each other, in particular:

1. Agents use a different mechanism to merge folder trees. 
2. Agents don't exchange the full list of files' availability with each other. This has its own drawbacks, see below. 
3. Use the SHA1 hashing algorithm. 
4. Agents don't use file signatures.

These changes also imply some limitations on compatibility between Agents v3.0.0 and older ones as mentioned below. 

Update from old versions, for already created and active jobs:
Current sync jobs will continue working as they are. Consolidation and distribution jobs will also continue working as configured. For them, it's possible to enable RAM optimization between the runs (only if all Agents in the job are updated to v3.0.0), and the setting will be applied to the next run. Synchronization jobs must be recreated, and the Ram-optimized option is also available only after all Agents are updated to v3.0.0.

If only some of the Agents in a job are updated to 3.0.0, they will run in compatibility mode with Agents of previous versions. This will improve RAM consumption on such agents, but not to the full extent.

Compatibility mode - is basically a set of custom parameters that provides better performance for agents of different versions. 

If all agents are eventually updated to a new version, they will still run in the compatibility mode. RAM consumption will be greatly reduced however to unleash the full power of RAM optimization, a new job should be created, see chart below.
Synchronization job, 10M files, enabled synchronization of NTFS permissions, between two Agents on Windows Server 2019 and 2022.

A newly created job 

There is a checkbox to make a fully RAM-optimized job. If it's selected, only v3 agents can be added to such a job.

Later this cannot be changed in Sync jobs. It can be changed for Consolidation or Distribution jobs, however changes will be applied on the next job run only.

Enabling this option is actually a transparent instruction for the agent to use a set of parameters for RAM optimization. They cannot be changed manually later, even by infusing such parameters through Profile.

If the option is checked, Agents of older version cannot be added to the job and will be grayed out on the selection. 
Once the job is created, it's mentioned whether it's a filly ram-optimized or not on the job configuration overview: 

Limitations and peculiarities 

Even though RAM optimization option have its benefits of reducing the RAM requirements, it has some side effects: 

 - the Agent does not know its upload and download difference with others. It only knows the files that it is currently uploading or downloading. Thus, for example, upload and download queue in Agent UI, and Active files in job run details in the Management Console may be empty at a given moment.

 - the Agent won't be able to inform about pending uploads to the central server. However, this can be workarounded with a custom parameter "merge_full_availability": true in Job Profile. 

- RAM consumption may be higher than expected during folder tree merge between the agents.