Shared Snapshots for EC2's Elastic Block Store Volumes

Today we are adding a new feature which significantly improves the flexibility of EC2's Elastic Block Store  (EBS) snapshot facility. You now have the ability to share your snapshots with other EC2 customers using a new set of fine-grained access controls. You can keep the snapshot to yourself (the default), share it with a list of EC2 customers, or share it publicly. Here's a visual overview of the data flow (in this diagram, the word Partner refers to anyone that you choose to share your data with):

The Amazon Elastic Block Store lets you create block storage volumes in sizes ranging from 1 GB to 1 TB. You can create empty volumes or you can pre-populate them using one of our Public Data Sets. Once created, you attach each volume to an EC2 instance and then reference it like any other file system. The new volumes are ready in seconds. Last week I created a 180 GB volume from a Public Data Set, attached  it to my instance, and started examining it, all in about 15 seconds.

You can use the AWS Management Console, the command line tools, or the EC2 API to create a snapshot backup of an EBS volume at any time. The snapshots are stored in Amazon S3. Once created, a snapshot can be used to create a new EBS volume in the same AWS region. Sharing these snapshots, as we are now letting you do, makes it possible for other users to create an identical copy of the volume.

The new ModifySnapshotAttribute function gives you the ability to set and change the createPermission attribute on any of your snapshots. We've also added the ResetSnapshotAttribute function to clear snapshot attributes and the DescribeSnapshotAttribute function to get the value of a particular attribute.

The DescribeSnapshots function now lists all of the snapshots that have been shared with you. You can also use this function to retrieve a list of all of our Public Data Sets.

You can also modify snapshot permissions using the AWS Management Console:

How can you use this? Off the top of my head, here are a number of ideas:

1. If you are a teacher or professor, create and share a volume of reference data for use in a classroom setting (and take a look at the AWS in Education program too).
2. If you are a researcher, share your data and your results with your colleagues, both within your own organization and at other organizations.
3. If you are a developer, share your development and test environments with your teammates. Snapshot the environments before each release to make it easy to regenerate the environment later for regression tests.
4. If you are a business, you can use snapshots to store data internally, with external clients, or with partners. This could be reference data, results of a lengthy and expensive computation, a set of test cases (and expected results) or even a set of pre-populated database tables.

I'm sure you have some ideas of your own; please feel free to share them in a comment!

Update:Shlomo Swidler posted some really good ideas in his Cloud Developer Tips blog.

As is often the case with AWS, we'll use this new feature as the basis for even more functionality later.

-- Jeff;

Comments

[Shlomo]

I've been looking forward to this feature!

Here's an article in which I describe some more cool things you can do with shared snapshots, and also some predictions about products and services we'll see that are based on this feature.

http://clouddevelopertips.blogspot.com/2009/09/cool-things-you-can-do-with-shared-ebs.html

[Max]

This is a super idea! Ideal for sharing datasets in big public company like ours where each subsidiary has at least one Amazon account of their own.

[Valerie]

Apologies for posting this here - but it IS the AWS blog - so I have to say:
WHY ISN"T THE AWS CHALLENGE FORM ACCEPTING SUBMISSIONS???

I mean jeez guys, cmon!! We've been working hard at this, we're way inside the deadline, and your system is just giving us an indeterminate 'there was an error submitting your data' - which is BS, everything is fine with OUR data.

If this is how AWS functions then it probably can't even handle the real, incredible application we've submitted.

Give us an alternate route to submit it, seriously!!

[www.anvilon.com]

Be careful when you create shared EBS volumes! There are hidden dangers lurking:

http://alestic.com/2009/09/ec2-public-ebs-danger

(The above post includes a $100 reward to help demonstrate the risks.)

[Sergio]

This is a great feature, thanks!

It would be even greater :-) if you could specify read only permissions on the shared snapshot. Any plans?

[Shlomo]

Hey Jeff,

Any chance of upgrading the Import to S3 feature to allow importing directly to an EBS snapshot? It would save a lot of drudgery, splitting large files and reconstituting them after import. You could also offer the ability to separate the data onto multiple snapshots.