Introduction
Rsync
is an open-source application that provides fast incremental or mirror backup by leveraging built-in data deduplication algorithms. It is included in almost all Linux distributions.
Features
- Can update whole directory trees and filesystems
- Optionally preserves symbolic links, hard links, file ownership, permissions, devices and times
- Requires no special privileges to install
- Internal pipelining reduces latency for multiple files
- Can use rsh, ssh or direct sockets as the transport
- Supports anonymous rsync which is ideal for mirroring
Basic Use
|
|
Option | Description |
---|---|
-v | Verbose mode, outputs detailed information |
-r | Recurse into directories |
-a | Archive mode (-rlptgoD, no -A, -X, -U, -N, -H) |
-h | Output numbers in a human-readable format |
-z | Use compression to transfer data |
-e | Specify the remote shell to use |
-P | Same as –partial –progress |
-W | Copy files whole (without delta-xfer algorithm) |
–progress | Show progress during transfer |
–delete | Delete extraneous files from destination directories |
–remove-source-files | Sender removes synchronized files (non-dir) |
–exclude=PATTERN | Exclude files matching PATTERN |
–include=PATTERN | Don’t exclude files matching PATTERN |
–copy-as=USER[:GROUP] | Specify user and optional group for the copy |
–chmod=CHMOD | Affect file and/or directory permissions |
Samples
|
|
Copy /home/user/data/
(as source directory) to /mnt/backup
(as destination directory), using the options -avh
.
Rsync Server Setup
Rsync
can be used not only for local backups, but also to transfer data between different OSes (e.g. Windows, macOS, Linux, etc.). It can push data to a remote host (for backup) or pull from a remote host (for restore). However, an rsync
daemon needs to be set up on one side.
1. Set up Rsync Daemon on OpenWRT (Typical)
Install Rsync
Daemon:
|
|
Two files are required:
File | Description |
---|---|
/etc/rsyncd.conf | Rsyncd config file |
/etc/rsyncd.secrets | Rsyncd authentication password file |
For the rsyncd
config file:
|
|
Two directory folders named Tools
and Media
are set up for sharing, and the users user01
and user02
are specified for each share.
For the rsyncd
authentication password:
|
|
This defines two users and their passwords.
(OPTIONAL) Generating a Random Password with ‘OpenSSL’
To generate a pseudo-random password, we can use the OpenSSL via syntax:
|
|
-base64 option is used for encoding the output with length 30 characters.
Assign permissions to the password file:
|
|
This avoids the error "@ERROR: auth failed on module"
for security.
Restart the rsyncd
service:
|
|
Or reboot.
2. Set up Rsync Daemon on OMV (OpenMediaVault)
To enable the Rsync Daemon Server:
Add a share folder named “Media”
Create an account for authorization
3. Set up Rsync Daemon on DSM (Synology)
Enable rsync service
Go to Control Panel > File Services > rsync, and tick Enable rsync service.
Enabling rsync Accounts
- Go to Control Panel > File Services > rsync, and tick Enable rsync account.
- Click Edit rsync Account > Add to configure users
Assign user "rsync" to directory and privileges
Rsync Client Setup
Rsync
is not only useful for local backups, but it can also be used to transfer data between different OS such as Windows, macOS, and Linux. You can use PUSH
(backup to remote) or PULL
(backup to local), but a server daemon is required.
1. Set up Rsync client on OpenWRT (Typical)
To install the rsync
service:
|
|
To backup using interactive mode:
|
|
Alternatively, create a Rsync
password file to store each user’s password to avoid having to insert the password each time:
|
|
The file stored password is “P@ssw0rd01”.
You can create other password files as you have more than one user. For example, /etc/rsync.passwd2
, and assign the file 600
permission:
|
|
To upload local data to a remote server, use the following syntax:
|
|
To download files from remote to local:
|
|
2. Set up Rsync client on OMV (OpenMediaVault)
In this screenshot, the remote folder’s data (Media) is being downloaded to the local harddisk.
3. Set up Rsync client on Windows
Install Rsync
client software like a Linux through Chocolatey:
|
|
Use the following syntax to transfer files:
|
|
Schedule a shell script to run using a CRON job on Linux
Create a shell script:
|
|
Assign execute permission to the shell script:
|
|
Edit the crontab file to schedule the job:
|
|
Add the following syntax to the end of the file:
|
|
3. Others
Create a copy of the directory structure from the 2023-2024/ directory in the 2024-2025 directory
|
|
Option | Description |
---|---|
-a | “archive” mode |
–include ‘*/’ | includes all directories (*/) |
–exclude ‘*’ | excludes all files (*) |
2023-2024/ | Existing Folder |
2024-2025 | New Folder |
Conclusion
Rsync
is a useful backup tool that provides fast incremental file transfer and supports various types of operating systems.Finally, I strongly recommend using the ‘
-e ssh
’ option to transfer data with public key, as it provides better security through an encrypted tunnel.
Reference
- rsync (samba.org)
- Rsync | DSM - Synology Knowledge Center
- RSync — openmediavault 6.x.y documentation
- Linux 使用 rsync 遠端檔案同步與備份工具教學與範例
- rsyncd in openWRT | #Villa’s syslog (villasyslog.net)