Clone Data Script

This script automates the process of backing up files from a remote server to a local directory. It uses rsync over SSH to ensure that the local backup directory is synchronized with the remote directory. The script includes logging capabilities and retry mechanisms in case of connection issues.

BACKUPS_HOST: The hostname or IP address of the backup server (e.g., backup.elosys.net).
BACKUP_DIR: The local directory where backups will be stored (e.g., /home/amine/backups).
LOGFILE: The file where logs will be stored (default: $BACKUP_DIR/out.log).

1. Logging Setup

touch $LOGFILE
exec 2>&1
exec >> $LOGFILE

Creates the log file if it doesn't exist.
Redirects both stderr and stdout to the log file for comprehensive logging.

2. Date Variable

DATE="date +\%Y-\%m-\%d\ \%H:\%M:\%S"

Defines a variable DATE to format and print the current date and time.

3. Start Log Entry

echo "["$(eval $DATE)"] Clone backup start"

Logs the start time of the backup process.

4. Checking Remote Backups Availability

while ! ssh -p 2224 $BACKUPS_HOST stat /upload/*.* \> /dev/null ;
do
    echo "["$(eval $DATE)"] Cannot contact backup server. Retrying in 1mn"
    sleep 30
done
sleep 2

Uses a while loop to repeatedly check if backups exist on the remote server.
If the server is not reachable, logs a message and retries after 30 seconds.
Introduces a short delay after the server becomes reachable.

5. Synchronizing Backup

rsync -ahvz -H --delete --exclude='out.log' --exclude=".*" --exclude='anonymization' -e 'ssh -p 2224' $BACKUPS_HOST:/upload/ $BACKUP_DIR

Uses rsync to synchronize the remote backup directory (/upload/) with the local backup directory ($BACKUP_DIR).
Options used:
- -a : Archive mode, which preserves permissions and other attributes.
- -h : Human-readable output.
- -v : Verbose mode.
- -z : Compression during transfer.
- -H : Preserve hard links.
- –delete: Delete files in the local directory that are not in the remote directory.
- –exclude: Exclude certain files or directories from being copied.
- -e 'ssh -p 2224': Use SSH with a specified port for the connection.