Configuring Linux OS cache servers

Linux cache server utilizes FUSE based solution to provide access to files to 3rd party applications that are not physically present on cache server. Together with Ganesha NFS server it allows to access this virtual file system over NFS v3 or v4. Linux cache server is used in in a File cache or Hybrid work jobs. 

Resilio Agent installation
Third party packages installation
Configure Ganesha NFS server
Configure NFS Clients accessing Ganesha NFS exports
Configure File caching/Hybrid work jobs
Configure Samba server
Configure Samba clients accessing the samba shares
Applying any job changes or restarting the Gateway Agent

Resilio Agent installation

Install Resilio Agent via package or tarball unpacking. 

Third party packages installation

Resilio Agent allows the exposure of gateway shares using either NFS (via Ganesha NFS server) or Samba (via Samba server). The specific requirements for additional libraries depend on the chosen configuration:
- NFS and Samba setup: Certain libraries must be installed regardless of whether NFS or Samba is used.
- NFS-Specific setup: Additional dependencies may be required for NFS functionality. Ensure that all necessary libraries are installed based on your environment to guarantee proper operation of the gateway shares.

apt install -y software-properties-common
apt update && sudo apt-get upgrade -y
apt install -y fuse3 libfuse2 nfs-ganesha nfs-ganesha-vfs  

setenforce 0
dnf install centos-release-nfs-ganesha5
dnf install fuse3
dnf download libnfsidmap.x86_64 libntirpc.x86_64 libprometheus-cpp.x86_64 libtirpc.x86_64 libwbclient.x86_64 nfs-ganesha.x86_64 nfs-ganesha-vfs.x86_64 nfs-utils.x86_64 rpcbind.x86_64
rpm -i --nodeps libnfsidmap.x86_64
rpm -i --nodeps libntirpc.x86_64
rpm -i --nodeps libprometheus-cpp.x86_64
rpm -i --nodeps libtirpc.x86_64
rpm -i --nodeps libwbclient.x86_64
rpm -i --nodeps nfs-ganesha.x86_64
rpm -i --nodeps nfs-ganesha-vfs.x86_64
rpm -i --nodeps nfs-utils.x86_64
rpm -i --nodeps rpcbind.x86_64

setenforce 0
wget https://internal.resilio.com/install/centos-release-nfs-ganesha5.rpm
rpm -i --nodeps centos-release-nfs-ganesha5.rpm
dnf download libnfsidmap.x86_64 libntirpc.x86_64 libprometheus-cpp.x86_64 libtirpc.x86_64 libwbclient.x86_64 nfs-ganesha.x86_64 nfs-ganesha-vfs.x86_64 nfs-utils.x86_64 rpcbind.x86_64
rpm -i --nodeps libnfsidmap.x86_64
rpm -i --nodeps libntirpc.x86_64
rpm -i --nodeps libprometheus-cpp.x86_64
rpm -i --nodeps libtirpc.x86_64
rpm -i --nodeps libwbclient.x86_64
rpm -i --nodeps nfs-ganesha.x86_64
rpm -i --nodeps nfs-ganesha-vfs.x86_64
rpm -i --nodeps nfs-utils.x86_64
rpm -i --nodeps rpcbind.x86_64

setenforce 0		
dnf install -y lsof fuse3
dnf install -y oracle-gluster-release-el8
dnf config-manager --enable ol8_addons ol8_UEKR6 ol8_appstream
dnf install -y glusterfs-server nfs-ganesha-vfs
setsebool -P ganesha_use_fusefs 1

setenforce 0
dnf install -y centos-release-nfs-ganesha5
dnf install -y libnfsidmap libntirpc libprometheus-cpp libtirpc libwbclient nfs-ganesha nfs-ganesha-vfs nfs-utils rpcbind

dnf install rpcbind nfs-utils libtirpc libwbclient libnfsidmap fuse3 fuse-libs
wget https://internal.resilio.com/install/centos-release-nfs-ganesha5.rpm
rpm -i --nodeps centos-release-nfs-ganesha5.rpm
mkdir packages
cd packages
dnf download nfs-ganesha nfs-ganesha-vfs libntirpc libprometheus-cpp
rpm -i --nodeps -v *.rpm

Configure Ganesha NFS server

Make export directory writable for all

chmod 0777 /mnt/ganesha-test

Ensure nfs-ganesha server is stopped, then edit the file /etc/ganesha/ganesha.conf. Use example config file for Ganesha as a template

NFSv4 {
  Only_Numeric_Owners = true;
      }
EXPORT
  {
  Export_Id = 12345;
  Path = /mnt/ganesha-test;
  Pseudo = /mnt/ganesha-test;
  Protocols = 3,4;
  Access_Type = RW;
  Squash = none;
  FSAL {
         Name = VFS;
       }
  }        

Configuration file supports several EXPORTs. For each new unique job location a new EXPORT must be added to the config. Export_Id, Path and Pseudo must be different for every export.

The folder that represents Resilio job does not have to be a root of exported folder, it may be a part of subtree.

Start nfs-ganesha to apply new configuration
systemctl restart nfs-ganesha

EXPORT
  {
  ...
  Attr_Expiration_Time = 5;
  ...
  }

Configure NFS clients accessing Ganesha NFS exports

Mount exported NFS folder from another Linux host

mount -v -o async,nconnect=16,noatime <server>:/mnt/ganesha-test /mnt/ganesha-test

Ensure read and write operations work normally and cause files to hydrate in Resilio virtual folder (number of local files in Management Console will grow every time you open a new file). Mount exported NFS folder on several Linux hosts and make sure Agent does not stuck under load.

Configure Samba server

The following guide is verified for Ubuntu and Debian distros. Install Samba server

apt install -y samba

Configure Samba. Add the following to /etc/samba/smb.conf, where path /mnt/storage/share1 is the Resilio job folder

[share1]
comment = any comment
read only = no
path = /mnt/storage/share1
guest ok = no

To ensure the config is valid, run as root:

testparm

Add samba users. This step can be skipped if you already have a configured Samba server with users' access to it. Note, the users must exist in /etc/passwd. Run the following command as root, it will prompt for a password for each user:

smbpasswd -a user1

Make sure shared folder is mounted:

# mount | grep resiliofs
resiliofs on /mnt/storage/share1 type fuse (rw,nosuid,nodev,relatime,user_id=0,group_id=0,default_permissions,allow_other)

Restart samba server:

systemctl restart smbd

Configure Samba clients accessing the samba shares

Mount configured folder from a Linux host:

USER=user1 PASSWD=mysecurepassword123 mount //my-samba-server/share1 /mnt/smb

Ensure read and write operations work normally and cause files to hydrate in Resilio virtual folder as number of local files in Management Console will grow every time you open a new file.

Configuring File caching/Hybrid work job

Linux cache server can be added to the job only after the Agent and 3rd party packages were installed and properly configured. Two locations must be selected for Caching Gateway agent.
Note, the selected paths cannot be changed later after the job is saved!  See here for more details.

Linux path: it is the "Path" matching Ganesha or Samba config file above, or a directory inside it. Is the file access location which is made available over NFS/Samba remotely. Files are virtually represented by this location for further accessing. This path must exist and be empty.
Linux cache: the actual file download location, not available for file access. 

.Xauthority
.Xauthority*

Applying any job changes or restarting the Gateway Agent

Should you need to apply any changes in the job or restart Resilio Agent or its host, please follow the pattern below:

• Stop any application / daemon using the NFS share exported by Ganesha
• (optional, just speeds things up) Unmount exported shares
• Stop nfs-ganesha daemon
• Stop Resilio Agent / restart host

Start all components in the opposite order:

• Start Resilio Agent first
• Start nfs-ganesha daemon
• Ensure exported servers are mounted on Linux machines that will consume the data
• Start your applications that will work with the data on Ganesha exports