⚡

A single Rsync backup server is a single point of failure—if it crashes, critical backups may be lost. Implementing an Rsync cluster with redundancy ensures that backups remain available even if a node fails.

This guide will walk you through setting up an Rsync high-availability cluster with step-by-step execution and expected output examples.

📌 What you’ll learn:
✅ How to set up multiple Rsync backup servers for redundancy
✅ How to automate data synchronization between backup nodes
✅ How to configure automatic failover using HAProxy
✅ How to monitor Rsync backups for performance and errors

🛑 1. Why Use an Rsync Cluster for Backups?

🔹 Single Point of Failure – If the backup server crashes, backups become inaccessible.
🔹 Scalability Issues – A single server may not handle thousands of backups efficiently.
🔹 No Real-Time Replication – If the backup fails before a sync is complete, the latest data could be lost.

✅ Solution: Set up an Rsync cluster that replicates backups across multiple nodes for high availability.

⚡ 2. Setting Up an Rsync High-Availability Cluster

This step-by-step setup includes:
✅ Primary Backup Server (backup1) – Main storage node
✅ Secondary Backup Server (backup2) – Redundant backup node
✅ Clients (site1, site2) – Systems sending backups
✅ HAProxy for automatic failover

🔹 2.1 Install Rsync on Both Backup Servers

Execute these commands on both backup1 and backup2:

sudo apt update && sudo apt install rsync -y  # Ubuntu/Debian
sudo yum install rsync -y  # CentOS/RHEL

📌 Expected Output:

Reading package lists... Done
Building dependency tree... Done
The following packages will be installed: rsync

✅ Create a backup directory on both servers:

sudo mkdir -p /backups/site1
sudo mkdir -p /backups/site2

✅ Set correct permissions:

sudo chown -R user:user /backups

📌 Ensures that Rsync can access the backup directory.

🔹 2.2 Enable Rsync Daemon for Cluster Syncing

✅ Edit the Rsync configuration file (/etc/rsyncd.conf) on both backup1 and backup2:

uid = root
gid = root
use chroot = yes
max connections = 10
log file = /var/log/rsyncd.log
pid file = /var/run/rsyncd.pid
lock file = /var/run/rsync.lock

[backup]
    path = /backups
    read only = no
    hosts allow = backup1.example.com backup2.example.com

📌 This allows both backup servers to sync data with each other.

✅ Start the Rsync daemon:

sudo systemctl enable rsync
sudo systemctl start rsync

✅ Verify Rsync is running:

sudo netstat -tunlp | grep rsync

📌 Expected Output:

tcp  0  0  0.0.0.0:873  0.0.0.0:*  LISTEN  1234/rsync

📌 Confirms that Rsync is listening for sync connections.

🔹 2.3 Sync Data Between Backup Nodes

✅ On backup1, sync to backup2:

rsync -avz --delete /backups/ backup2.example.com:/backups/

📌 Expected Output:

sending incremental file list
backups/
backups/site1/
backups/site2/
sent 234.56K bytes  received 1.23K bytes  50.45K bytes/sec

📌 Verifies that backup data is successfully transferred.

✅ On backup2, sync to backup1:

rsync -avz --delete /backups/ backup1.example.com:/backups/

✅ Automate the sync using cron:

crontab -e

✅ Run Rsync sync every hour:

0 * * * * rsync -avz --delete /backups/ backup2.example.com:/backups/

📌 Ensures that backups are synchronized hourly.

🔄 3. Implementing Automatic Failover for Rsync Cluster

If backup1 fails, backup2 should automatically take over.

🔹 3.1 Set Up DNS Failover (Simplest Method)

If backup1 goes offline, backup.example.com should point to backup2.

✅ Edit DNS records in your domain provider settings:

backup.example.com  ->  192.168.1.10 (Primary Backup Server)
backup2.example.com ->  192.168.1.11 (Secondary Backup Server)

📌 Use DNS failover (Cloudflare, AWS Route 53) to auto-switch if backup1 is down.

🔹 3.2 Configure HAProxy for Automatic Failover

✅ Install HAProxy on a separate failover server (failover.example.com):

sudo apt install haproxy -y

✅ Edit /etc/haproxy/haproxy.cfg:

frontend rsync_frontend
    bind *:873
    mode tcp
    default_backend rsync_backend

backend rsync_backend
    mode tcp
    balance leastconn
    server backup1 192.168.1.10:873 check
    server backup2 192.168.1.11:873 check backup

✅ Restart HAProxy to apply changes:

sudo systemctl restart haproxy

✅ Verify HAProxy failover:

curl http://failover.example.com:873

📌 Expected Output:

Rsync Daemon Active

📌 If backup1 is down, requests automatically route to backup2.

📊 4. Monitoring and Managing Rsync Clusters

🔹 4.1 Check Rsync Logs

✅ Monitor Rsync logs on backup1 and backup2:

tail -f /var/log/rsyncd.log

📌 Shows real-time sync operations and errors.

🔹 4.2 Monitor Rsync Cluster with Prometheus

✅ Install node_exporter for Rsync monitoring:

sudo apt install prometheus-node-exporter -y

✅ Check Rsync node performance:

curl http://localhost:9100/metrics

📌 Expected Output:

# HELP node_network_transmit_bytes_total Total number of bytes transmitted
node_network_transmit_bytes_total 123456789

📌 Now, visualize Rsync cluster status in Grafana.

🛠️ 5. Troubleshooting Rsync Cluster Issues

✅ Test failover by manually stopping Rsync on backup1:

sudo systemctl stop rsync

📌 Ensure backup2 takes over automatically.

📊 6. Summary

✅ An Rsync cluster ensures reliable, failover-ready backups.

💬 Join the Discussion!

Do you use high-availability Rsync clusters?
What failover strategies do you prefer?

💬 Share your experience in the comments below! 🚀

👉 Next Up: Scalable Rsync Backup Solutions for Enterprise Environments