Downloading Files on a Dodgy Network Connection.

I was on holiday this weekend and the place we stayed didn't have a strong internet connection. It was good enough for browsing, but not good enough for multiple people streaming stuff off of the internet. So we wanted to pull down some files from our media library (typically large files) to where we were staying.

I had a web server at home, and my computer with us on holiday.

I copied files to a folder on the server that was offered up to the internet.

The two problems with large files and dodgy connections is that at any time, the network can break. Browsers can sometimes resume downloads but experience tells me you have to do something a bit more resilient.

Stackoverflow pointed me in the direction of "lftp". LFTP is a sophisticated file transfer program supporting a number of network protocols (ftp, http, sftp, fish, torrent).

To install on mac with brew installed, it was as simple as:

1brew install lsftp

...and now we script it like this:

1lftp -c 'set net:idle 10
2         set ssl:verify-certificate false
3         set net:max-retries 0
4         set net:reconnect-interval-base 3
5         set net:reconnect-interval-max 3
6         pget -n 10 -c "https://myhost.com/iistemp/bigfile.zip"'

Good luck!