It's important to encrypt our data at rest, and the most common way to do this in AWS is to use the AWS Key Management Service (KMS). When we request data that is encrypted with a KMS Key, we need to perform an API request to KMS to decrypt the data using our KMS Key. Here's a screenshot of the costs of these requests within the eu-west-1 region:
As you can see, if we encrypt all our data across multiple services where the data is frequently accessed, we could potentially end up with a substantial KMS bill. This is where caching comes to the rescue, and various AWS services handle caching of KMS Keys in different ways. I'll highlight a few of my favourite AWS services below and try to help you understand what to expect when working with them in combination with AWS KMS.
Amazon S3
In 2020 AWS introduced S3 Bucket Keys, which is a way to reduced the requests to the KMS API for objects encrypted by Server Side Encryption with KMS (SSE-KMS). When Bucket Keys are enabled S3 will generate a short lived key for each requestor which is stored locally within S3.
It's worth noting that requests for objects in S3 which existed before enabling the use of Bucket Keys will not use the bucket keys, and may incur additional costs.
DynamoDB stores a copy of the KMS key used to encrypt the data in the DynamoDB table, tied to the specific connection, and stores it locally for 5 minutes. If a request for data decryption is made outside the 5 minutes another call is made to the KMS API.
You can learn more about DynamoDB encryption at rest here:
Messages in SQS are encrypted once received by the queue, and are only decrypted by the consumer. SQS uses data keys local to SQS, which facilitates the reduction in requests to the KMS API. The encryption and decryption data keys are separate keys, but adhere to the same lifecycle defined by you. The period can be set between 1 minute and 24 hours, with a default period of 5 minutes; the shorter the time period the more secure, and the longer the time period the more cost effective as you'll need less requests to the KMS API.
You can learn more about SQS integration with KMS here:
As you can see most of the caching capabilities are handled in a similar fashion, but there are distinct differences between each service highlighted above. It's worth understanding what is possible before committing to a particular method of handling encryption at rest.
With services like SQS allowing data keys to last for up to 24 hours, and the default time period only being 5 minutes, you should understand the data being passed through the queue and what your risk appetite is for that data, before making a decision on the lifecycle of the data keys.
If you liked this post, you may also be interested in...
