Don't overwrite newer files on destination agents in distribution jobs

Overview
Cloud storage specifics
Limitations

Overview

The transfer_job_skip_newer_files parameter is a custom job profile setting that allows destination Agents to skip downloading files if their local version is newer. 

By default, this parameter is disabled, meaning all files from the source will overwrite files on the destination. Add custom parameter to Job profile before starting the job run
transfer_job_skip_newer_files:true

When enabled, destination Agent compares mtime (modified time) of files from source and destination.

If the destination file has newer mtime, the file is skipped and not transferred to the destination.

If mtime is equal, ctime and file size are additionally compared with those from the source:
 a) they match — file is treated as identical, it's not re-trasnferred, but it's also not considered as skipped. 
 b) either of these doesn't match — file is skipped, counted as "skipped" in logs.

In the job run

• File counters in the job run are reduced accordingly by the number of skipped files, and do not raise errors.
• Triggers, apart from "Before finalizing download" are supported.
• Skipped files remain on the source if the job is configured to remove files from source after transfer.
• Skipped files are reported as "not delivered" in job run report.
• Files skipped by timestamp are reported in the "Total files to transfer" overview.

• Log entry "Path skipped" is written for each skipped file in the job run's logs.
  

Cloud Storage Specifics

In cloud destinations where mtime can't be set directly, Resilio Agents use internal algorithm to determine if a file is "newer" or not:
 - file considered to be the same and will be overwritten on cloud destination if its timestamp is equal or newer and size is the same.
 - file considered to be newer and won’t be overwritten (will be skipped) on cloud destination if its timestamp is newer and size is different.

Limitations

Multiple Destinations
Functionality is not optimized to work with multiple destinations in Distribution job. Management Console invalidates such job runs, both in UI and API, giving the error on start.
Starting job run with an API call gives generic error “The job <job name> contains configuration errors, please fix them before starting the job”. Recheck the destination Agents in the job configuration.

Job triggers
Functionality does not work with “Before finalizing download” trigger in the job, and all files are overwritten on destination.

Job configuration
Files that are skipped by destination are not cleared from source after transfer, in case the job is configured so.
High availability groups are not supported.

Job run
The skipped files are reported in Total files to transfer in job run overview.