To transfer a large amount of data (200GB) across a 100mbps network without saturating the connection, I used rsync with the –bwlimit=KBps flag.
Transferring from Windows to Ubuntu 9.04, I first mounted the windows share:
sudo mkdir /mnt/winshare sudo mount -t smbfs -o username=user,password=pass //winserver/share /mnt/winshare
Next, run rsync with a specified bwlimit. Optionally, unmount the share
sudo rsync -vrR --delete --delete-excluded --bwlimit=6144 /mnt/winshare/* /destination/ sudo umount /mnt/winshare
Note that the bwlimit is in kilobytes not kilobits — 6144 KB ~= 48 Mbps. Also note that the bwlimit is limiting the I/O bandwidth…whatever that means.
--bwlimit=KBPS limit I/O bandwidth; KBytes per second
NTop (http://www.ntop.org) is great for tracking bandwidth usage:
sudo apt-get update sudo apt-get install ntop
Set the administrative account password for ntop to run under:
sudo ntop -A
Start the service:
sudo /etc/init.d/ntop start
Browse the reports on the default port, 3000. http://yourserver:3000/
I think you made a mistake here… unless both my math (admittedly horrible) rsync runs are fooling me too. From above:
"Note that the bwlimit is in kilobytes not kilobits — 6144 KB ~= 48 Mbps. Also note that the bwlimit is limiting the I/O bandwidth…whatever that means."
6144 is ~= 6mbps not 48.
I'm trying to use no more than 30% 100Mb connection so I'm using
–bwlimit=30000 (thirty thousand) and the end result is showing around 10Mb/s for a few hundred Mb.
@ezekieldas: 8 bits in a byte, so 6144 kiloBYTES ~= 6 megaBYTES ~= 48 megaBITS (bits per second is usually measured in round numbers; 1 kilobit is 1000 bits. 1 kilobyte is 1024 bytes, so there is often a slight ambiguity when converting bits to bytes.)
So, if you are trying to limit your sync to 30 megabits per second, your number should be roughly (30 / 8) * 1024 = 3840 kilobytes.
Unless my math is bad…