Amazon S3

Earlier today we rolled out Amazon S3, our reliable, highly scalable, low-latency data storage service.

Using SOAP and REST interfaces, developers can easily store any number of blocks of data in S3. Each block can be up to 5 GB in length, and is associated with a user-defined key and additional key/value metadata pairs. Further, each block is protected by an ACL (Access Control List) allowing the developer to keep the data private, share it for reading, or share it for reading and writing, as desired.

The system was designed to provide a data availability factor of 99.99%; all data is transparently stored in multiple locations.

S3 is a very cost-effective data storage solution. Using S3's economical pay-as-you-go model, storing 1 GB of data for 1 month costs just 15 cents. Transferring data in and out of the system costs 20 cents per GB. As Colin Faulkingham notes, there's also a BitTorrent interface.

I've got to catch a plane to Silicon Valley in a few minutes, or I'd write a lot more. You can read more at TechCrunch, Business Week, and at Scripting News. We are looking forward to seeing developers create all sorts of cool applications with this new system.

If you are in Silicon Valley, stop by and see us at SD Expo on Wednesday or Thursday.

-- Jeff;

Comments

[Simon B]

Two things that would be good:
1. Provide a payment platform so that the content provider can charge the downloader for the bandwidth used, instead of the publisher paying the bill.
2. rsync upload.

[Brian Klug]

From the Amazon S3 design principles:

"Simplicity: The system should be made as simple as possible (– but no simpler)"

How can something be made more simple then what is impossible? Was that a joke?

[Paul Noeldner]

What's missing is some standard web service client drivers to install an S3 connection in Windows and Unix operating systems. It should look just like another disk drive (kind of like a USB driver does). That would really be neat and make it a lot more usable.

[Jens Alfke]

I'm unclear what the definition of "developer" is, here. I can think of lots of client-side apps I could write to use this, but every user of the app would need to sign up for a "developer" account to use it, since I have no desire to be stuck with the fees and have to pass them on to the users.

Brian: Einstein famously said "Things should be made as simple as possible, but no simpler."

Paul: Writing filesystem drivers is nasty work, and highly platform dependent (Win / Linux / BSD / Mac OS). It would be a lot easier to interface the S3 protocols to WebDAV, since every OS already knows how to mount a WebDAV store as a pseudo-filesystem.

[Brandon Hopkins]

Bandwidth and storage space are cheap...I don't know how many people are going to want to pay for storage...

[Jacob Levy]

Jeff

S3 looks really cool. To make large objects useful (e.g. for implementing your own storage layer) it would be great if you could add an API to write into, or read from, an arbitrary location of an existing object. Something like the unix lseek+write or lseek+read APIs. Then I could implement my own NFS like system, or use it to store live mysql databases, etc.

[Len]

For those who interested in simplicity: the slogan above is quoting Albert Einstein out of context. Originaly the famous phrase came out of this line: "...In my opinion the theory here is the logically simplest relativistic field theory that is at all possible ..." http://www.spaceandmotion.com/Most-Simple-Scientific-Theory-Reality.htm

[Tony.h]

I've just signed up a friend to amazon S3 and i must say is the coolest simple storage system, i highly recommend anyone needing such a service to give it a try