Avoiding a Success Disaster

For a while I have been using the term "success disaster" to characterize what can happen on the web all too easily. What's a success disaster? You put up a piece of content somewhere and you get ready to handle a reasonable number of downloads.

Being the creative person that you are, however, links to your content shows up on Digg and Slashdot the same day that you are written up on TechCrunch. Suddenly the whole world wants in, and unless you've been Slashdotted before, you have no idea how to respond. Your basic assumptions about server capacity, network traffic, and data transfer limitations have just been blown out the window.

Getting more hardware to address this peak demand is probably not the right thing to do. All that traffic will  go away just as quickly as it came, and you don't want to raise your monthly burn rate just to accomodate these infrequent peaks in demand.

You need a shock absorber to help you deal with this transient surge in attention.

Think of it this way. If 200 of your closest friends suddenly showed up on your doorstep and said that they'd be hanging around for a while, you probably wouldn't drop everything and build an extension to your house. Instead, you would use an on-demand resource, in this case the nearest hotel, to handle this (hopefully transient) need for more room.

The folks over at the Spanning Sync Blog found themselves in just this situation a few days ago. After putting a new video online, traffic surged and they were saturating their server's network connection. They quickly moved the video over to Amazon S3 and the downloads proceeded very smoothly. In fact they served up over 6,000 11,726 copies of the video in just one day - more info in the comments.

For more information, read their post: 5,000 Video Downloads. Time for Amazon S3.

-- Jeff;

Comments

[Charlie Wood]

Thanks for the link, Jeff. In fact, we served up 11,726 copies of that movie yesterday. And when I look at my S3 account page I see that Amazon handled over 35 GB of traffic for us yesterday and charged us a whopping $7.04. Nice. We'll be sticking with S3 for storage, and will look at EC2 for processing power next.

-Charlie