InvXim

May 23, 2009

Download a Corrupt File? Fix it with BitTorrent

Filed under: The Internets — Tags: — Elvedin @ 4:38 pm

I have been working on a few virtual machine images on my remote server and I downloaded them using my fast link at work. When I came home, I found that the ZIP file I compressed the images in would not extract, most likely due to my laptop locking up during the download. Rather than downloading a few gigabytes over my home’s 150KB/s link, I decided to be more efficient.

Since I have root on the remote system, I was able to install the Debian distribution BitTorrent client and tracker while using the µTorrentclient on my laptop. Although any public or private tracker would do, even no tracker if you’re using µTorrent, I decided to set up the tracker on the remote system. To do that, we use the command

bttrack --port 6969 --dfile /tmp/dfile &

The ‘&’ character at the end means that it will detach from your console and run in the background. The next step is making the .torrent file. This process consist of the application splitting your file(s) into logical chunks and making a hash of each chunk for integrity checks after you download each chunk. You use the btmakemetafile command on the remote system which has the valid file like so;

btmakemetafile filename http://127.0.0.1:6969/announce

where “filename” is the name of the file and 127.0.0.1 is the IP or hostname of your tracker. Since the .torrent file has been made, now we can seed it on the remote system with following command

btdownloadheadless filename.torrent

This will do its own integrity check before it begins seeding.

Finally, download the .torrent file on your client system and open it with the BitTorrent client. Save the file from the BitTorrent client to the same place where it was originally downloaded. Make sure no “skip hash check” options are checked. When this is done correctly, the BitTorrent client will do the integrity check of your downloaded file(s) and mark “unfinished” those chunks that were invalid. By this time, you should be connected to the seeder on the remote system and downloading the chunks that were invalid. My .torrent’s chunk size was 1 MB and there was only one chunk I had to download. Compared to downloading a few gigabytes, I have saved quite a bit of time by only downloading one megabyte.

If you opted to not use a valid tracker URL when making your .torrent, now is your chance to go into µTorrent’s (or other clients) options and click on the “Peers” tab, right click on an empty space and then “Add Peers”. If you don’t know which port the BitTorrent client is running on the remote system, on the remote system do the following –


ps aux |grep btdownloadheadless
netstat -nap |grep PID_of_BitTorrent_client

The first command returns all of the running processes on the system and pipes the output to grep which only shows the line that has “btdownloadheadless” in it. The second column from that output would be the PID. The second command runs netstat and runs the PID and port information, then we pipe it to grep which only returns the lines that have the PID of the BitTorrent client. This may return some false positives, which is why you should check the very last column which will be in the form of “PID/process name” (this means you can skip the ‘ps aux’ step and replace the PID with the process name in the netstat part). To find the port in the output of netstat, look at the 4th column which will have the IP/hostname and then the port prefixed with the ‘:’ character. Ignore the leading ‘::’ and copy the value of that column until the end of the port, and paste it into µTorrent. This will directly connect to a seed so now you can get the chunks from the remote system without a tracker.

There are other worthwhile uses of BitTorrent, including the ROCKS Cluster distribution of install files for compute nodes through BitTorrent. Have your own examples? Please share them in the comment section below.

Powered by WordPress