Fast File Transfer: Moving beyond acceleration with data optimization

posted in: FileCatalyst | 2

As mentioned a short time ago, we have some material from our old blog that we are slowly migrating to the new one. Being that we are in the business of fast file transfer, it made since to bring back one of our more popular articles on the subject. Although written around the time of FileCatalyst 2.0 (2.7.x is the current generation), the information is still relevant, and the article continues to be widely read by visitors to the main site. Here it is, for readers of the new blog.

Original article written by Chris Bailey on July 31, 2007 9:53 PM


Fast File Transfer: Moving beyond acceleration with data optimization

If you are in the market for a fast file transfer solution, you have a couple of options. First there is the traditional FTP client software, some of which use parallel TCP streams to speed up your transfers. Then there are the UDP based transfer applications, FileCatalyst is one of them. These applications can maximize the data across your internet connection regardless of network conditions. If you have a T3, you will get exactly T3 speeds. The other approach to fast file transfer is data optimization.

By reducing the data that needs to be transmitted, you can effectively transfer the file faster. Even if the actual data going across your line is not optimal, you may still get faster rates because of the data reduction ratio. Consider a database file, or large spreadsheet. Since this data is highly compressible, you can reduce transfer time significantly just by zipping it up.

Another way to optimize file transfers is to send only portions of a file that have changed. Consider the database mentioned above. You need to back up this database on a daily basis to a remote location over a T1. The database file is 2 GB. If you maximize your T1, it could take almost 3 hours to transfer the file each day, even with the best acceleration on the market. But what if only 100MB had changed in the file? If you could detect and transfer only the portion that has changed you would reduce the transfer time by a factor of 20. Now the transfer only takes 9 minutes!

But hold on, that database is probably compressible as well, so even with only a 2:1 compression ratio you could cut that transfer time in half again. So now it is only 4 and a half minutes, or 40 times faster than your link speed!

FileCatalyst 2.0 was released earlier this year and does acceleration, as well as differencing and compression. It does it for you in the background, so there is no wait time to compress the file prior to transferring; it is all done on the fly, from one automated tool. With FileCatalyst, file transfers are as fast as your link, i.e. T1, T3, etc… The only question is how much faster it will go beyond that speed. That depends whether you have transferred the file previously and whether the data is compressible or not.

fast file transfer
Fast File Transfer

The table above lists some speed gain examples. Of course there are a lot of cases that do not benefit in any way from this technology, but there are just as many that do. FileCatalyst should be considered an option in either case since it offers the best of all worlds; that is, industry leading acceleration as well as data optimization. You can always be assured you are getting the fastest possible file transfer with FileCatalyst.

To read more about accelerating and optimizing file transfer with the FileCatalyst family of products visit

2 Responses

  1. EDI

    The way data is transmitted online with fast internet connections today, I have given up looking for fast file transfer programs. FTP is all I need today.

    • Greg

      Many people, especially if they are transferring small files across the city, will not see dramatic benefits from acceleration technologies. It’s important to note, though, that the kind of accelerated transfer we offer isn’t really meant to be consumer-oriented. When you’re an organization with a 155Mbps (or even a 1Gbps) connection, transferring large files over any network with medium to high latency, the gains are dramatic. When an enterprise can send a file in 2 hours that would have taken 2 days with FTP, that’s when they pull the trigger on buying an acceleration solution. 🙂

      Speaking about our product rather than acceleration in general for a moment, we’ve had a few sales to corporations for whom acceleration was a “bonus,” and what they really wanted were the other features (ie. automation, delta transfers, Java web applets) we include.

      You’ve got absolutely the right idea, though– don’t fix what ain’t broke. If FTP is working for you and you’ve never felt the “pain” that an acceleration solution heals, then FTP is exactly right for you!