As of the end of 2011, there are 762 billion (762,000,000,000) objects in Amazon S3. We process over 500,000 requests per second for these objects at peak times.
Here's the annual growth chart:
This represents year-over-year growth of 192%; S3 grew faster last year than it did in any year since it launched in 2006.
Where are all of these objects coming from? Although we definitely made it easier for you to delete objects using Multi-Object Deletion and Object Expiration, we also gave you plenty of ways to upload new objects using Multipart upload, AWS Direct Connect, and AWS Import/Export.
As you can imagine, building, running, and adding new features to a system as large and as complex as S3 is no simple task. Here are some of the open positions on the S3 team:
- Software Development Engineer
- Senior Software Development Engineer
- Systems Engineer
- Senior Product Manager
- Director
-- Jeff;
Jeff,
That's awesome. Is this a replicated or un-replicated object count? I am assuming full replication?
Posted by: Randybias | January 30, 2012 at 09:19 PM
Congrats to Amazon S3.
Posted by: Bucket Explorer | January 30, 2012 at 09:24 PM
Awesome! Amazon S3 is extremely scalable and robust store for backup and long term archiving. No wonder this level of growth is being seen.
Posted by: Zmanda | January 31, 2012 at 06:40 PM
Jeff, quick question in regards to this data. Does these numbers include the Amazon Cloud Storage service? If then all of this isn't storage isn't paid for since you get 5GB free worth of storage. What I would like to know is how many objects are being stored outside of anything related to Amazon's Cloud Drive or Kindle Cloud storage.
Posted by: Scott Shepard | February 01, 2012 at 02:11 AM
Scott - We don't break down the numbers to that level of detail.
Posted by: Jeff Barr | February 01, 2012 at 05:19 AM
When I plug this into a basic logistic growth model (A Verhulst-style curve fits nicely - I'm assuming the 2006 number is artificially low because that was a short year.), there's tremendous uncertainty about the long-term because you're so early in the curve and the exponential effects dominate. Still, an 'eyeball' fit suggests the object count should grow to at least 1,000,000 billion (1.0E+15) and not start to roll off (stabilize) before 2024. I interpret that as a lower bound, and that growth could run much longer, and reach much higher numbers. It implies doubling or more every year for at least the next 10 years. 2+ trillion objects Q4 2012, 5.6 trillion in 2013. Seems like a herculean infrastructure challenge. Is it even possible relative to world-wide disk drive production?
Posted by: Spike McLarty | February 01, 2012 at 07:27 AM
I wonder how much of this growth is due to Dropbox ? Can anyone provide a guess ?
Posted by: H D Bolak | February 02, 2012 at 01:42 PM
HD - We would only share this if Dropbox decided to talk about it. So you'll have to talk to them.
Posted by: Jeff Barr | February 02, 2012 at 02:48 PM
Check out Navitas CloudXport - a versatile data transportation solution that helps customers move data seamlessly to-and-from the Amazon Web Services (AWS) Simple Storage Service (S3) by providing an easy means to access S3 as a shared folder or a drive on any computer. CloudXport is a standards based solution that allows customers to use built-in file explorer tools on Windows, UNIX, or MacOS computers and do not require any additional software installation on client machines. Since customers retain their AWS credentials and encryption keys on-premise and take control and ownership of data they can rest assured their data are secure. CloudXport can be used to access S3 in US-GovCloud region. Please visit http://navitas-tech.com/Products/cloudxPort.aspx for more details.
Posted by: Srini Bayireddy | February 06, 2012 at 10:16 AM
Hi Jeff. Good work! Are you able to reveal how much data that is in terms of bytes? I'm wondering things like... what fraction of the global output of hard disk drives are you using?
Posted by: James | February 26, 2012 at 02:58 AM