Adding the Export to AWS Import/Export

I blogged about the new AWS Import/Export feature this past spring and told you how it allows you to load any amount of data into Amazon S3 by simply shipping the data to us on a compatible storage device. The response to that announcement has been excellent and our customers are now sending us terabytes of data every week.

Today I'd like to tell you about the new Export aspect of this feature. Using a workflow similar to the one you'd use to import data, you prepare a MANIFEST file, email it to us, receive a job identifier in return, and then send us one or more specially prepared storage devices. We'll take the devices, verify them against your manifest file, copy the data from one or more S3 buckets to your device(s) and ship them back to you.

A "specially prepared" storage device contains a SIGNATURE file. The file uniquely identifies the Import/Export job and also authenticates your request.

You can use the new CREATE EXPORT PLAN email command to simplify the process of exporting a data set that won't fit on a single storage device. Given the block size, and the device capacity (either formatted or unformatted), the command returns a link to a zip file with a set of MANIFEST files inside. 

You will be charged a fixed fee of $80.00 per device and $2.49 per hour for the time spent copying the data to your device. Normal charges for S3 requests also apply. There is no charge for bandwidth.

There are many uses for this new feature. Here are some ideas to get you started:

• Disaster Recovery - If your local storage fails or is destroyed, use the Export feature to retrieve your precious data.
• Data Retrieval - After creating a large data set (either by gathering it up or by computing it) in the cloud, use the Export feature to get a local copy.
• Data Distribution - Take a large data set, sell copies, and use the Export feature to take care of the distribution.
• Data Processing - Use the Import feature to load a large data set (yours or a customers') into the cloud, do some computationally intensive processing (e.g. de-duplication), and then get the data back using the Export feature.

Sign up here to get started with AWS Import/Export.

What are your ideas? How can you use this new feature? Leave a comment if you'd like!

-- Jeff;

PS - We haven't forgotten our international users! We’re working on a number of solutions to enable international shipments to and from the US, and to enable support in the EU region.

Comments

[Shlomo]

This is great - it closes the loop.

How about exporting a snapshot of an EBS volume?

[Rich Bruklis]

I have a quick question about the Export feature for Disaster Recovery. How does one retrieve data?

If I have to send you a tape cartridge or hard drive or other media, do you copy it back to that device and send it overnight?

If that is the method and let's say it is 5 years later, that media might be totally obsolete (i.e. a tape cartridge that has been upgraded or completely retired).

Just curious. I am thinking about DR in terms of RTO and RPO.

Thanks
Rich

[Colin Percival]

When will you "close the loop" and integrate AWS Export with the retail side of amazon.com so that I can order a drive and have it arrive filled with my data rather than needing to ship it back and forth three times? For anyone who wants their data quickly -- especially once you add support for international users -- this could be a major benefit.

[Maarten]

One obvious use would be to keep offsite backups of data stored in S3.

And of course disaster recovery in extreme cases (using S3 as an offsite backup), but for that purpose having the option to buy a drive from Amazon instead of shipping your own might be better since it can save several days.

[Mark McAndrew]

Dear Jeff,

If AWS can make profitable use of several million volunteered home PCs (assumed-insecure nodes, of course, but incredibly cheap), then you might like to take a quick look at this.

The Charity Engine launches later this year and, with the backing of ten international charities, should quickly become the largest volunteer grid in the world. All it needs now are the founding customers.

Obviously, highly sensitive apps will never be suitable. Public-funded 'big science' projects (climate change, clean energy, particle physics, et al) are ideal, but of greater relevance here is The Charity Engine's massive public cloud becoming a powerful new option for AWS customers.

As The Charity Engine has negligible overheads, it can provide CPU/hours for 'less than cost' (ie. below the price of the electricity they consumed). It will therefore be stunningly cheap to use, at around 1c per CPU/hour or even less. A 1000 CPU/hour job for $10, a 1m CPU/hour job for $10k. These are order-of-magnitude cost savings.

If there is indeed a market for truly vast amounts of dirt-cheap, embarrassingly parallel processing and/or VERY distributed storage, then we are about to witness the ultimate evolution of the cloud paradigm - a world wide computer.

If AWS would like to be involved, we would welcome the opportunity to meet up and discuss the matter in more detail.

Best regards,

Mark McAndrew
The Charity Engine
UK

PS. Please forgive us contacting you via your blog like this. It is because we've found many spam filters reject the word 'charity'. A sad state of affairs, I'm sure you'll agree.

[Ted]

Congratulations! Our selection committee compiled an exclusive list of the Top 100 cloud computing Blogs, and yours was included! Check it out at http://thedailyreviewer.com/top/cloud-computing

You can claim your Top 100 Blogs Award Badge at http://thedailyreviewer.com/pages/badges

Cheers!

[Steven Roussey]

If data could fit on a BlueRay disc, that would be a nice option.

[anonymous]

How about subscribing to periodic exports on to disk. As suggested here; new harddrives, blueray, whatever works.

The use-case here would be physical backups.