Amazon S3 Bucket Policies - Another Way to Protect Your Content

Update August 22, 2011:

We are aware of the existence of a tool that scans S3 looking for buckets that allow anonymous users READ permission. Bucket level READ permissions only allow an user to list the objects within a bucket. However, users with the ability to list could probe into the bucket looking for unprotected content, potentially resulting in undesirable access to content as well as usage charges. We have inspected the permissions of all S3 buckets and have sent an email to the owner of buckets that appear to have excessively permissive access controls granting the READ permission for anonymous users. We have also emailed the owners of all buckets that grant the WRITE or WRITE ACL permission to anonymous users. With WRITE permissions granted to Everyone, an anonymous user can access, modify or delete any object in a bucket.

We strongly encourage you to inspect and, if necessary, restrict the permissions on your buckets and on the objects in each bucket. If you are concerned about the integrity of your objects, you can inspect the modification dates in the buckets. You can also inspect the Server Access Logs for the buckets in question. The easiest way to secure your bucket is by using the AWS Management Console. First select a bucket and click the ”Properties” option within the “Actions” drop down box. Now select the “Permissions” tab of the “Properties” panel. Verify that there is no grant for “Everyone” or “Authenticated Users”. If one or both exist, remove the “Everyone” and “Authenticated Users” grantees. Your bucket will now be inaccessible to anonymous users.

Earlier this year we launched a popular feature enabling our users host static websites on S3. Bucket level READ permissions for everyone – the permission configuration we’ve warned our users about - is not required for S3 website hosting. However, S3 website hosting does require READ permissions at a per-object level. To verify that your bucket is secured against anonymous operations, use the instructions above. To verify individual objects have anonymous READ, you can use the S3 Console to view the permissions on individual objects and verify that Everyone is granted READ permission. Granting anonymous READ on every object is commonly done using ACLs but can also be done using Bucket Policies.

Users of Amazon S3 have been looking for additional ways to control access to their content. We've got something new (and very powerful), and I'll get to it in a moment. But first, I'd like to review the existing access control mechanisms to make sure that you have enough information to choose the best option for your application.

The two existing access control mechanisms are query string authentication and access control lists or ACLs.

The query string authentication mechanism gives you the ability to create a URL that is valid for a limited amount of time. You simply create a URL that references one of your S3 objects, specify an expiration time for the query, and then compute a signature using your private key.

The Access Control List (ACL) mechanism allows you to selectively grant certain permissions (read, write, read ACL, and write ACL) to a list of grantees. The list of grantees can include the object's owner, specific AWS account holders, anyone with an AWS account, or to the public at large.

Each of these mechanisms controls access to individual S3 objects.

Today, we are adding support for Bucket Policies. Bucket policies provide access control management for Amazon S3 buckets and for the objects in them using a single unified mechanism. The policies are expressed in our Access Policy Language (introduced last year to regulate access to Amazon SQS queues) and enable centralized management of permissions.

Unlike ACLs which can only be used to add (grant) permissions on individual objects, policies can either add or deny permissions across all (or a subset) of the objects within a single bucket. You can use regular expression operators on Amazon resource names ("arns") and other values, so that you can control access to groups that begin with a common prefix or end with a given extension such as ".html".

Policies also introduce new ways to restrict access to resources based on the request. Policies can include references to IP addresses, IP address ranges in CIDR notation, dates, user agents, the HTTP referrer, and transports (http and https).

Finally, with bucket policies we have expanded your ability to control access based on specific S3 operations such as GetObject, GetObjectVersion, DeleteObject, or DeleteBucket. 

When you put all of this together, you can create policies that give you an incredible amount of access control.

You could set up a bucket policy to do any or all of the following:

• Allow write access...
• To a particular S3 bucket...
• Only from your corporate network...
• During business hours...
• From your custom application (as identified by a user agent string).

You can grant one application limited read and write access, but allow another to create and delete buckets as well. You could allow several field offices to store their daily reports in a single bucket, allowing each office to write only to a certain set of names (e.g. "Nevada/*" or "Utah/*" and only from the office's IP address range).

Policies and ACLs interact in a well-defined way and you can choose to use either one (or both) to control access to your content. You can also convert your existing ACLs to bucket policies if you'd like.

Read more in the new Using Bucket Policies section of the Amazon S3 Developer Guide. We'll also be holding an Introduction to Bucket Policies webcast on July 13th.

What do you think? How will you use this exciting and powerful new feature?

-- Jeff;

Comments

[Adrian Cole]

I think this is exciting news. Great work!

[Kin Lane]

This is great. I was updating all of our servers to centralize the storage of files on Amazon S3.

Except many of the heavy files our clients upload are accessed through different web servers. Some are public...some are private.

Each client usually has its own machine. I wanted a better way to grant access to various buckets from different Amazon EC2 instances and / or public IP addresses.

Will play around with this more.

[Maryloutyler]

In light of todays security announcement from Amazon - 08.22.2011 - Important Security Notification regarding your Amazon S3 bucket setting, for those users who unknowingly have the privileges set incorrectly, like me, can you post a comment about what a bucket should look like in terms of privileges in order to prevent unauthorized access?

My suspicion is that the EVERYONE user set to READ is what is causing the unauthorized access, but all the documentation I've read is not really helpful in what we, as layman users, need to set our privileges at in order to protect ourselves.

Can you assist? Many thanks in advance.

[chris collins]

+1 for to the question from Maryloutyler. Also if you are using S3 as a static website is there any way to prevent the security issue that has been raised.

[Robert Morris]

+1 to the question posed by Maryloutyler.

[Mike]

I also received the email from Amazon this morning, but not sure what to do.. Any suggestions?

[Jeff]

+1 I agree with the above and the statement by Maryloutyler. I have 150+ videos that I have behind a membership site and need to protect the content from people snooping around looking for freebies.

[Mike]

Hi Jeff, I saw your comment above about having a membership site. I am going to have a commercial video site created soon and am still exploring ideas/ways to implement it. How does yours work? Is AWS the best choice for selling your streaming videos online?

[APhillips]

You can add me to the list of folks who have LOTS of videos behind a membership site as well as numerous training videos for my virtual staff. I'm not really clear how to protect these videos.

[Mike]

How should "wrong" look and how should "right" look? The Amazon email elegantly and generally tells me nothing. Your blog post helps a bit, but is not clear. Like others, I can't see what might be wrong or how to fix it. Thank you.

[Sylvia]

+1 to many of the above. I use my S3 buckets for all my website videos, TV programming, squeeze pages, etc. Since I cannot know who is going to land on a squeeze page for my Internet marketing business arm, it is tough to set specific user or condition or timing policies. People around the world have "business hours" at different times. I'm a lay-person w/o knowledge of coding. Just barely understood the X's and checkmarks to allow permissions.

[Pedro Soto]

+1 to the above. you have just alerted me to a danger and left me clueless as to how to resolve it, in a way that I can understand and implement.

[JohnR]

S3Media Stream is a plugin for WordPress and Joomla that enables playing private streaming videos and audio, whether they are in a membership system or not.
That way, you can set your videos and audios to private, yet play them with the plugin who creates expiring URLs on the fly.
Wordpress version: www.wp21century.com
Joomla 1.5: www.joomla21century.com

I hope this is useful?

[Cary Abramoff]

It boggles my mind the s3 servers cannot allow public read buckets to prevent list mode. Why not have the presence of an index.html file for example prevent the default behavior being list. There are many reasons to wish to have a bucket be public read but that does not mean one want to list the contents. Or perhaps having url referrer control the listing.

[DJ]

One way to protect your s3 bucket is to set the folder itself to private, but keep all the files inside as public. Members and customers can access their media while keeping your bucket from getting scraped by tools like this or other users.