rsync, parallel

Basic Form of rsync

rsync -rv --progress local_dir destination/

• -r: recursive
• -v: verbose

Transfer a directory from local PC to an EBS attached to an ec2 instance

rsync -rv --progress -e "ssh -i ~/.ssh/your-ssh-key" local_dir destination/

• -e is for inputting credential

(example)
rsync -rv --progress -e "ssh -i ~/.ssh/daesoo-linux-pc.pem" SoundlySpeech_24000hz ec2-user@ec2-13-60-224-245.eu-north-1.compute.amazonaws.com:/workspace/

---

Basic Form of parallel

parallel echo "First: {1} Second: {2}" ::: A ::: X Y Z
output:

First: A Second: X
First: A Second: Y
First: A Second: Z

rsync with parallel

parallel -j 12 --eta rsync -rv {1} {2} ::: local_dir ::: destination/

• -j: number of jobs
• --eta: estimated time of arrival

(example)
parallel -j 12 --eta rsync -rv -e {1} {2} {3} ::: "ssh -i ~/.ssh/daesoo-linux-pc.pem" ::: SoundlySpeech_24000hz ::: ec2-user@ec2-13-60-224-245.eu-north-1.compute.amazonaws.com:/workspace/