Increasing Amazon S3 Data Transfer Performance

The Amazon S3 team is now beta-testing support for an important low-level networking feature which has the potential to significantly increase the performance of large data transfers to and from S3, particularly (but not limited to) for long distance data transfers.

In particular, there is a new beta endpoint which supports the RFC 1323 model for TCP window scaling. With this option in effect, a larger amount of data can be in transit across the network at any given time, reducing the impact of speed-of-light delays as data is broken down in to chunks, sent across the internet, verified, and acknowledged. When the two nodes in question are far apart, the time that it takes for the data to travel from sender to receiver (and for the acknowledgments to travel back) turns out to be just as important as the raw data transfer rate.

Per the thread in the Amazon S3 Forum, early results from the beta testers are quite good with reported speedups of 4x to 18x! If you are moving large amounts of data into or out of S3 then you will definitely want to implement this feature.

Note that we are supporting this through a beta endpoint. The endpoint will be present for the duration of the test and will then vanish. Of course you won't want to embed this endpoint in any shrink-wrapped software.

You will need to dig deep into your operating system's networking setting in order to take advantage of this new feature. For Linux systems the information here will get you started (there's also some information for other operating systems in there too). Windows 2000 and Windows 2003 users can start at this Knowledge Base article. I know nothing about the Mac except that my youngest daughter keeps asking me for one.

I would also recommend that you read this very informative article by Brian Tierney for further background information and some formulas and recommendations on how to calculate and set the optimal window size.

-- Jeff;

PS - Did you know that Amazon.com sells actual windows?

Comments

[Robert Accettura]

Mac OS X 10.5 automatically adjusts it's network settings making this not really necessary for most people:

Self-Tuning TCP

Let Leopard adjust TCP buffer size automatically. Get optimum application performance, especially in high-bandwidth/high-latency environments.

http://www.apple.com/macosx/features/300.html#networking

You can also see this page for windows users:
http://www.dslreports.com/tweaks/MTU

[Kin Lane]

Excellent guy! Thanks so much for this. 90% of the time the performance levels are acceptable.

However I have lost a couple of clients who stated in initial prototyping that the performance was not adequate for their needs.

I am eager to test out more. Thanks for noticing and your work on this.

[Peepz]

Well, Window Scaling occasionally does hurt performance when one of the routers/termination devices/proxies/ssl applicances does not understand the options.

Linux: http://kerneltrap.org/node/6723
Windows: http://8help.osu.edu/3253.html

Finally, even if scaling were used and receiver buffer isnt sized appropriately (Windows has 17KB by default which is per RTT throughput) then it might not get all the benefits.

Dont get me wrong, RFC 1323 is great but just wanted to share my past experiences of having issues with them

[Indus Khaitan]

This is excellent. Also, in one of our implementation scenario on text data objects, we saw increased S3 performance by adding an in memory lazy local cache.

BTW, Jeff, any insight into SimpleDB beta? When are the invites going out?

Regards.