New - Amazon S3 Server Side Encryption for Data at Rest

A lot of technical tasks that seem simple in theory are often very complex to implement. For example, let's say that you want to encrypt all of the data that you store in Amazon S3. You need to choose an encryption algorithm, create and store keys (while keeping the keys themselves safe from prying eyes), and "bottleneck" your code to ensure that encryption happens as part of every PUT operation and decryption happens as part of every GET operation. You must take care to store the keys in durable fashion, lest you lose them along with access to your encrypted data.

In order to save you from going through all of this trouble (and to let you focus on your next killer app), we have implemented Server Side Encryption (SSE) for Amazon S3 to make it easier for you to store your data in encrypted form. You can now request encrypted storage when you store a new object in Amazon S3 or when you copy an existing object. We believe that this important (and often-requested) new feature will be welcomed by our enterprise customers, perhaps as part of an overall strategy to encrypt sensitive data for regulatory or compliance reasons.

Amazon S3 Server Side Encryption handles all encryption, decryption, and key management in a totally transparent fashion. When you PUT an object and request encryption (in an HTTP header supplied as part of the PUT), we generate a unique key, encrypt your data with the key, and then encrypt the key with a master key. For added protection, keys are stored in hosts that are separate and distinct from those used to store your data. Here's a diagram of the PUT process for a request that specifies SSE:

Decryption of the encrypted data requires no effort on your part. When you GET an encrypted object, we fetch and decrypt the key, and then use it to decrypt your data. We also include an extra header in the response to the GET to let you know that the data was stored in encrypted form in Amazon S3.

We encrypt your data using 256-bit AES encryption, also known as AES-256, one of the strongest block ciphers available. You can apply encryption to data stored using Amazon S3's Standard or Reduced Redundancy Storage options. The entire encryption, key management, and decryption process is inspected and verified internally on a regular basis as part of our existing audit process.

You can use Amazon S3's bucket policies to allow, mandate, or forbid encryption at the bucket or object level. You can use the AWS Management Console to upload and access encrypted objects.

To learn more, check out the Using Encryption section of the Amazon S3 Developer Guide.

-- Jeff;

PS - There's no additional charge for SSE.

Comments

[R]

The master key means that the data can be decrypted in response to (at least) legal requests, correct?

[R]

Ah, nevermind. Clearly the data can be decrypted by AWS; that's what happens whenever you GET it. But this could give you peace of mind/protection if someone walked off with a hard drive from an S3 datacenter, or got unauthorized access to some part of AWS infrastructure but not the key management (not that Amazon would let either of those things happen).

Relatedly, and more practically, it's probably really useful when a company needs to meet the letter of a requirement to encrypt data at rest.

[Max C]

I have a lot of valuables at home, but they're safe because the house has great locks. And I leave the keys next to the house rather than carry them around, so it's really convenient too. I think I'm the sort of person who will like SSE.

[Tim Chiu]

Can you share some insight on how decryption would work and how one would share the decryption key with another tool or collaborator?

[Mojozoox Me]

Great! This is what everyone wants, hmm... I guess? But what happens of the Key Database is lost somehow.

[Greg Cain]

Nice!
I'd really love to see this extended to RDS, to give us encrypted Data at Rest in a MySQL database - please, please, please!

[Jeff Barr]

Tim - The encryption and decryption are transparent. There's no need to share keys.

[Jeff Barr]

Max - I don't think that's a good analogy. The keys are stored in encrypted form, and are not simply laying around.

[Alecco]

This is very interesting.

It would be nice to have better details on how it's implemented. Using AES256 tells us very little. Does it use CBC? What kind of padding and IV?

Can you be more specific on what scenarios this feature will protect the stored data?

Thanks!

[S3 Browser Team]

That's great news!

Just wanted to let you know that S3 Browser Freeware now comes with full support for Server-Side Encryption.

Using S3 Browser, you can easily:
– Check whether the file is encrypted or not
– Encrypt and decrypt existing files or whole S3 Bucket.
– Automatically encrypt files during uploading.

More details can be found here: http://s3browser.com/amazon-s3-server-side-encryption.php

--
Ivan,
S3 Browser Team

[Tim Chiu]

Hi Jeff,

If we ever need to invalidate a key, will there be an API that allows us to do so? Also, what is the performance impact on PUTs and GETs when one turns on SSE?

Thanks,
Tim

[Jannick]

Seems to me that this (somewhat) addresses the issue of Amazon/S3 being compromised, since an attacker would gain access to both the data, key and master key. It does however not provide any additional security for the scenario where customer EC2 hosted apps are compromised, as the data will be decrypted on request.

It would be really nice if the GET and PUT requests becomes capable of accepting public/private keys to be used for encrypting/decrypting the Encrypt Key.

[Jannick]

Correction: "would need to get access to"

[Carl]

I would love to see more detailed information on how the key encryption works. If AWS is a black box, and data still goes into and comes out of the box in the clear, then I'm not understanding how it's more secure. Is it to address someone breaking into AWS and trying to steal your data? Because if the keys are stored AWS-side, they'd have those too, right?

[Ole_morten]

If my S3 keys are exposed (accont id and secret), can these be used to fetch all my data, not encrypted?

[Mark]

@Ole yes. Nothing really is changing about how you use S3. Except that you can instruct S3 to encrypt the bits before/for storage. The point is to encrypt data at rest. So if you compromise your AWS Keys that have full access to get data from S3, yes.

[Alex]

Same question as Ole. If this is all transparent, and someone gets a hold of my id and secret, they can get all my data? In that case, what's the point of encryption?

[Alex M.]

Jeff, if that isn't a good analogy, then can you explain how the master key is secured? In the marketecture diagram it's in the same green box with the encryption key. So... it's like putting my keys in one of *these* next to the house?

http://www.amazon.com/dp/B000ES7GKK

[Nuutti Kotivuori]

Is there a security whitepaper describing the processes keeping the encryption keys? Is there a separation between system administrators of the key-keeping servers and the data-keeping servers? Is this encryption compliant with PCI requirements of keeping card holder data encrypted? Will this be a part a of your next PCI audit scope?

[Michael]

Could someone explain me goal of SSE? Client-side encryption resolves absolutely clear security issues and challenges. What security issues can be solved by SSE if all my content can be downloaded after getting my S3 credentials (account ID and secret key).

Thanks.

[Jeff Barr]

@Alecco - We use AES/CBS/PKCS5Padding as the encryption algorithm. We use a unique encryption key to encrypt each object. This key is encrypted with a master key which is regularly rotated.

SSE protects your data in attack scenarios where an attacker has access to the data, but not access to both keys. Thus this protects against stolen/lost disks.

[Jeff Barr]

@Tim - We use a unique encryption key to encrypt each object. If you want to invalidate an encryption key used on an encrypted object, you can make a COPY request on that encrypted object. The new target object will now be encrypted with a new encryption key, if you chose to use server side encryption during the COPY operation. This effectively invalidates the original key and rotates your key to use a new key..

We have not seen any significant impact on performance when using SSE. Network or Internet latency will still remain the biggest determinant on faster throughput. This throughput performance may vary from customer to customer. We recommend you benchmark your requests to determine any performance impact.

[Jeff Barr]

@Jannick - We are constantly iterating on our products based on customer feedback. We appreciate the feedback and will gladly take this into consideration in future planning.

[Jeff Barr]

@Carl - Amazon S3 will decrypt the data only for requests from customers who have access permissions to download the data. Any other unauthorized requests will be denied. You can use ACLs, bucket policies to control who will have access to your encrypted data. Encrypted objects, encryption keys, and the master encryption key are stored on different hosts as an additional safeguard. Data cannot be decrypted without all 3 elements.

Yes, one of the uses of encryption is to protect against physical loss of a disk. While we secure customer data (read the AWS Security White Paper at http://awsmedia.s3.amazonaws.com/pdf/AWS_Security_Whitepaper.pdf for more information) we have heard from some of our customers that they would like the additional protection of encryption.

[Jeff Barr]

@Michael, @Alex -

SSE protects your data in attack scenarios where an attacker has access to the data, but not access to both keys. Thus this protects against stolen/lost disks. You can’t decrypt without having all the 3 elements: encrypted data, encryption key and the master key.

AWS will decrypt the object for valid requests from AWS customers who are allowed access to the data. Thus, customers still would be required to securely maintain and manage their access id and secret keys. Also, you can control who will have access to your data through ACLs and bucket policies.