You may recall that a couple of weeks ago, I mentioned that that Squirt, a regex compliant command line FTP client, was falling over when handling very large files. I have finally found the time to take a look into this and finding a solution proved to be a lot easier than I expected.
In short, what the get and put functions now do is generate a list of files to squirt and then processes each file in its own FTP instance. This approach does make the app a little slower when handling multiple small files, but this is a cost worth paying for the massively increased reliability.
The code is a bit of a mess at the moment, and in dire need of some refactoring, but it can be found at GitHub if you are interested in how the workaround works.