New - Amazon S3 Server Side Encryption for Data at Rest

A lot of technical tasks that seem simple in theory are often very complex to implement. For example, let's say that you want to encrypt all of the data that you store in Amazon S3. You need to choose an encryption algorithm, create and store keys (while keeping the keys themselves safe from prying eyes), and "bottleneck" your code to ensure that encryption happens as part of every PUT operation and decryption happens as part of every GET operation. You must take care to store the keys in durable fashion, lest you lose them along with access to your encrypted data.

In order to save you from going through all of this trouble (and to let you focus on your next killer app), we have implemented Server Side Encryption (SSE) for Amazon S3 to make it easier for you to store your data in encrypted form. You can now request encrypted storage when you store a new object in Amazon S3 or when you copy an existing object. We believe that this important (and often-requested) new feature will be welcomed by our enterprise customers, perhaps as part of an overall strategy to encrypt sensitive data for regulatory or compliance reasons.

Amazon S3 Server Side Encryption handles all encryption, decryption, and key management in a totally transparent fashion. When you PUT an object and request encryption (in an HTTP header supplied as part of the PUT), we generate a unique key, encrypt your data with the key, and then encrypt the key with a master key. For added protection, keys are stored in hosts that are separate and distinct from those used to store your data. Here's a diagram of the PUT process for a request that specifies SSE:

Decryption of the encrypted data requires no effort on your part. When you GET an encrypted object, we fetch and decrypt the key, and then use it to decrypt your data. We also include an extra header in the response to the GET to let you know that the data was stored in encrypted form in Amazon S3.

We encrypt your data using 256-bit AES encryption, also known as AES-256, one of the strongest block ciphers available. You can apply encryption to data stored using Amazon S3's Standard or Reduced Redundancy Storage options. The entire encryption, key management, and decryption process is inspected and verified internally on a regular basis as part of our existing audit process.

You can use Amazon S3's bucket policies to allow, mandate, or forbid encryption at the bucket or object level. You can use the AWS Management Console to upload and access encrypted objects.

To learn more, check out the Using Encryption section of the Amazon S3 Developer Guide.

-- Jeff;

PS - There's no additional charge for SSE.

Comments

[R]

The master key means that the data can be decrypted in response to (at least) legal requests, correct?

[R]

Ah, nevermind. Clearly the data can be decrypted by AWS; that's what happens whenever you GET it. But this could give you peace of mind/protection if someone walked off with a hard drive from an S3 datacenter, or got unauthorized access to some part of AWS infrastructure but not the key management (not that Amazon would let either of those things happen).

Relatedly, and more practically, it's probably really useful when a company needs to meet the letter of a requirement to encrypt data at rest.

[Max C]

I have a lot of valuables at home, but they're safe because the house has great locks. And I leave the keys next to the house rather than carry them around, so it's really convenient too. I think I'm the sort of person who will like SSE.

[Tim Chiu]

Can you share some insight on how decryption would work and how one would share the decryption key with another tool or collaborator?

[Mojozoox Me]

Great! This is what everyone wants, hmm... I guess? But what happens of the Key Database is lost somehow.

[Greg Cain]

Nice!
I'd really love to see this extended to RDS, to give us encrypted Data at Rest in a MySQL database - please, please, please!

[Jeff Barr]

Tim - The encryption and decryption are transparent. There's no need to share keys.

[Jeff Barr]

Max - I don't think that's a good analogy. The keys are stored in encrypted form, and are not simply laying around.

[Alecco]

This is very interesting.

It would be nice to have better details on how it's implemented. Using AES256 tells us very little. Does it use CBC? What kind of padding and IV?

Can you be more specific on what scenarios this feature will protect the stored data?

Thanks!

[S3 Browser Team]

That's great news!

Just wanted to let you know that S3 Browser Freeware now comes with full support for Server-Side Encryption.

Using S3 Browser, you can easily:
– Check whether the file is encrypted or not
– Encrypt and decrypt existing files or whole S3 Bucket.
– Automatically encrypt files during uploading.

More details can be found here: http://s3browser.com/amazon-s3-server-side-encryption.php

--
Ivan,
S3 Browser Team

[Tim Chiu]

Hi Jeff,

If we ever need to invalidate a key, will there be an API that allows us to do so? Also, what is the performance impact on PUTs and GETs when one turns on SSE?

Thanks,
Tim

[Jannick]

Seems to me that this (somewhat) addresses the issue of Amazon/S3 being compromised, since an attacker would gain access to both the data, key and master key. It does however not provide any additional security for the scenario where customer EC2 hosted apps are compromised, as the data will be decrypted on request.

It would be really nice if the GET and PUT requests becomes capable of accepting public/private keys to be used for encrypting/decrypting the Encrypt Key.

[Jannick]

Correction: "would need to get access to"

[Carl]

I would love to see more detailed information on how the key encryption works. If AWS is a black box, and data still goes into and comes out of the box in the clear, then I'm not understanding how it's more secure. Is it to address someone breaking into AWS and trying to steal your data? Because if the keys are stored AWS-side, they'd have those too, right?

[Ole_morten]

If my S3 keys are exposed (accont id and secret), can these be used to fetch all my data, not encrypted?

[Mark]

@Ole yes. Nothing really is changing about how you use S3. Except that you can instruct S3 to encrypt the bits before/for storage. The point is to encrypt data at rest. So if you compromise your AWS Keys that have full access to get data from S3, yes.

[Alex]

Same question as Ole. If this is all transparent, and someone gets a hold of my id and secret, they can get all my data? In that case, what's the point of encryption?

[Alex M.]

Jeff, if that isn't a good analogy, then can you explain how the master key is secured? In the marketecture diagram it's in the same green box with the encryption key. So... it's like putting my keys in one of *these* next to the house?

http://www.amazon.com/dp/B000ES7GKK

[Nuutti Kotivuori]

Is there a security whitepaper describing the processes keeping the encryption keys? Is there a separation between system administrators of the key-keeping servers and the data-keeping servers? Is this encryption compliant with PCI requirements of keeping card holder data encrypted? Will this be a part a of your next PCI audit scope?

[Michael]

Could someone explain me goal of SSE? Client-side encryption resolves absolutely clear security issues and challenges. What security issues can be solved by SSE if all my content can be downloaded after getting my S3 credentials (account ID and secret key).

Thanks.

[Jeff Barr]

@Alecco - We use AES/CBS/PKCS5Padding as the encryption algorithm. We use a unique encryption key to encrypt each object. This key is encrypted with a master key which is regularly rotated.

SSE protects your data in attack scenarios where an attacker has access to the data, but not access to both keys. Thus this protects against stolen/lost disks.

[Jeff Barr]

@Tim - We use a unique encryption key to encrypt each object. If you want to invalidate an encryption key used on an encrypted object, you can make a COPY request on that encrypted object. The new target object will now be encrypted with a new encryption key, if you chose to use server side encryption during the COPY operation. This effectively invalidates the original key and rotates your key to use a new key..

We have not seen any significant impact on performance when using SSE. Network or Internet latency will still remain the biggest determinant on faster throughput. This throughput performance may vary from customer to customer. We recommend you benchmark your requests to determine any performance impact.

[Jeff Barr]

@Jannick - We are constantly iterating on our products based on customer feedback. We appreciate the feedback and will gladly take this into consideration in future planning.

[Jeff Barr]

@Carl - Amazon S3 will decrypt the data only for requests from customers who have access permissions to download the data. Any other unauthorized requests will be denied. You can use ACLs, bucket policies to control who will have access to your encrypted data. Encrypted objects, encryption keys, and the master encryption key are stored on different hosts as an additional safeguard. Data cannot be decrypted without all 3 elements.

Yes, one of the uses of encryption is to protect against physical loss of a disk. While we secure customer data (read the AWS Security White Paper at http://awsmedia.s3.amazonaws.com/pdf/AWS_Security_Whitepaper.pdf for more information) we have heard from some of our customers that they would like the additional protection of encryption.

[Jeff Barr]

@Michael, @Alex -

SSE protects your data in attack scenarios where an attacker has access to the data, but not access to both keys. Thus this protects against stolen/lost disks. You can’t decrypt without having all the 3 elements: encrypted data, encryption key and the master key.

AWS will decrypt the object for valid requests from AWS customers who are allowed access to the data. Thus, customers still would be required to securely maintain and manage their access id and secret keys. Also, you can control who will have access to your data through ACLs and bucket policies.

[Jeff Barr]

@Nuuti - We have a white paper that provides an overview on the security on AWS ( http://awsmedia.s3.amazonaws.com/pdf/AWS_Security_Whitepaper.pdf ). We plan to update this paper to include discussion on SSE.

Through SSE you can achieve encryption of data at rest. You are welcome to use your audit process to evaluate whether SSE encryption and key management policies matches your security and compliance requirements. You can learn more about SSE through the Using Server Side Encryption topic in the Amazon S3 Developer Guide.

[Richard Threadgill]

This seems to address an extremely specific use case - meeting regulatory requirements to encrypt data at rest - without really addressing the broader desire to armor that data. Max' analogy seems spot on, although I might take it a bit further - all of the doors and windows to the house are locked, but because that makes it a hassle to get in and out, we've put the keys to the lock in a small (unlocked) box next to the door. Let's look closely at an example - suppose I provide an AWS / S3 - based service. I would like for my customers to upload sensitive data to S3, which my service will then analyze. I would like to give my customers a customer-unique key, so that they can be confident that only I can read their data, without requiring that they deploy a piece of client-side software to perform the encryption, or writing a custom receiver through which they will upload their data (after all, that receiver would then hold all the keys, in which case it doesn't improve security after all, just moves the target around). As deployed, SSE solves the case where a thief physically removes disks from the amazon data center, but somehow does not get the key database. Which seems like a strange hypothetical case. As described, it doesn't address the case of someone simply copying my customer's access url and then asserting to s3 that they are a valid client trying to access data to which they are entitled.

So, in the short term, it would be helpful to be able to specify the key with the upload / download request, since that protects my customer from exposure in the event of someone attacking S3 successfully. In the longer term, it would be helpful to have genuine public key encryption, where my customer presents one key on data upload, knowing that only I can read what they've sent, and I present a different key on access, knowing that that specific customer had their key when they sent me their data.

[Gunnar]

This is all about compliance and especially with consideration of the lifecycle management of disks in the AWS data center - am I right, Jeff?

Consider what happens if a disk in the data center goes bad (probably a few every day?) and has to be swapped out. As likely as there is some policy in place to sanitize the disk before disposal, would you rather hope that the policy to properly destroy any remaining data is followed, or be safe in the knowledge that all data on it is encrypted and the key is nowhere near it? Call it the dumpster-diver insurance.

[vivek]

Jeff, I agree SSE is great if one wants protection against stolen/lost disks. But can't we achieve almost similar protection through client side encryption? I am assuming that a customer who is worried about disk theft must also be using client side encryption to secure data from AWS credentials theft. I couldn't understand the value add through SSE.

[erha]

@Jeff
About your statement:

"SSE protects your data in attack scenarios where an attacker has access to the data, but not access to both keys. Thus this protects against stolen/lost disks."

Do you mean somebody can stole the S3 Amazon disk? Is this a joke?

[Arshad Noor]

While AWS has definitely added a useful control to protect customer data from being breached due to stolen hardware, if AWS customers think that SSE can be used to pass data-security regulations such as PCI-DSS, 201 CMR 17.00 and their ilk, then they're in for a surprise.

One of the tenets of effective key-management is that you - the owner of the data - have to prove that your controls will allow you to prevent unauthorized people from gaining access to the key. In key-management terms, this implies that access control for encrypted-data (ciphertext) is separated from access-control for cryptographic keys.

Unfortunately, as I understand SSE from its description, this separation does not exist; access to the ciphertext file automatically includes access to the decryption key. (In case you think this isn't such a big issue, remember that TJX paid a fine of $115M for breaching encrypted credit-card numbers that were stored in a database, whose users had automatic access to the decryption key when they had access to the table-data).

If you're encrypting data for compliance to a data-security regulation, the only sure-fire way of proving control is to use client-side encryption while protecting the decryption key with appropriate key-management controls.

Two postings on the topic in the S3 forum:

Why not ALWAYS encrypt? - https://forums.aws.amazon.com/thread.jspa?messageID=284505񅝙

Protecting sensitive data in public clouds with new FOSS - https://forums.aws.amazon.com/thread.jspa?messageID=284504񅝘

[Ronak]

Great features. Bucket Explorer will coming soon with this features.
Using Bucket Explorer you can
- Check whether the particular file is encrypted or not.
– Encrypt and decrypt existing Objects,Objects containing in folder or Objects of whole S3 Bucket.