This content originally appeared on Level Up Coding - Medium and was authored by Dimitri Papaioannou
How to secure data at rest with encryption as a service
Has this ever happened to you? You receive a notification from your bank, credit card, or tax specialist saying that their system has been breached and some client information was accessed by “unauthorized” persons. They are taking it very seriously and are conducting an investigation and even though they do not know which accounts were affected, they think you will probably be fine.
Unfortunately, incidents like this are far too common and only the most spectacular ones ever make it to the news. Why does this happen? It happens because your sensitive customer data is sitting in some file system, database, or email server unencrypted. Although there are passwords to access all these storage systems, they can be stolen, and often some employees have unrestricted access to the data.
Under the circumstances, it is safe to assume that sooner or later any software system will be breached and its data accessed. When this happens, the best way to protect sensitive data is to make sure it is incomprehensible by encrypting it. You have also to ensure that the key used to encrypt the data is not stored in the same place as the data. This is where encryption as a service comes into play.
Amazon KMS and Encryption SDK
This article provides a good overview of data encryption and its challenges. Here, I will focus on implementation and demonstrate a Java Springboot application that uses the AWS Encryption SDK to encrypt data that lives in a relational database. Similar approaches can be pursued for data that lives in other kinds of data stores.
The AWS umbrella of cloud offerings provides encryption as a service via a component called KMS. There is also an abstraction sitting on top of KMS called AWS Encryption SDK. They are both very easy to use, provided you have an AWS account.
Setup Users and Keys
Navigating the AWS ecosystem can be somewhat daunting. This tutorial provides excellent step-by-step instructions on how to set up uses and secret keys.
AWS services as pay-as-you-go so a small amount will be charged every time you use a service. If you don’t want to set up an AWS account, at the end of this article I provide an alternative of setting up KMS using localstack. Localstack runs AWS services on your local computer and is a great way to test out the services before moving onto the cloud.
Either way, once you set up a secret key, you will be given an identifier for the key that looks like this:
aws:kms:us-east-1:000000000000:key/34d769e2–4b23–42a7-b9b3–35315c44a2e9
This is not the secret key but just a way to reference it. The secret key lives on the AWS server.
Implementation
We start by defining a Spring configuration for connecting to the Encryption SDK.
The KmsMasterKeyProvider is configured with the key identifier we obtained when setting up the key. We are going to supply this in the configurations file via an environment variable.
aws.key.arn=${AWS_KEY_ARN}The second bean, CryptoMaterialsManager, is responsible for the communication between our program and the AWS server. Because we will be encrypting several columns on a lot of records, it is essential that we set up a local cache to avoid doing a service call for each encrypt/decrypt request.
When the SDK attempts to connect to the remote server it will look for the environment variables AWS_ACCESS _KEY and AWS_SECRET_ACCESS_KEY to authenticate. If you have an AWS account, you must set these variables in the runtime environment.
Now we define a service to handle encryption and decryption of the data.
Behind the scenes, the SDK is going to send the master key for descriptions. The decrypted master key will be used to decrypt the relevant portion in memory.
Database Encryption
We’ve seen that implementing encryption and decryption is very easy using the AWS Encryption SDK. We now want to employ this simple service to encrypt data at rest and decrypt it on demand. For this example, we are going to use the highly popular relational database PostreSQL.
The plan of action is the following:
- Decide which fields need to be encrypted and annotate them.
2. Intercept any data entering the database, identify the sensitive fields, and encrypt them.
3. Intercept any data coming out of the database, identify the encrypted fields, and decrypt them.
The first step is to come up with a way to identify data fields to be encrypted. We will do this by defining an annotation:
We can use the annotation to mark any fields in an Entity as follows.
Note that the encrypted values are of type byte[] since this is the level at which the SDK encrypts data. We could convert these byte arrays to Strings and back but we are going to avoid it because often the encrypted data is much larger than the original data and it may overflow.
We will need a corresponding byte array column in the database. For Postgres this is BYTEA:
CREATE TABLE IF NOT EXISTS customer (
id uuid DEFAULT uuid_generate_v4() NOT NULL,
username VARCHAR NOT NULL,
ssn BYTEA,
phone_number BYTEA,
PRIMARY KEY(id)
);
The final step is to define interceptors to encrypt and decrypt as the data travels to and from the database. There are many ways to do it. One could use Spring AOP, but in this example, I am going to use Hibernate event listeners.
The following code handles the encryption part when a record is inserted or updated in the database.
The main complexity identifying the fields marked encrypted and obtain their values. I have delegated another class, AnnotatedFieldProvider, which I omit here for brevity but can be inspected on GitHub along with the rest of the code.
The following code handles the decryption part when records are read from the database.
Testing and Continuous Integration
To complete the code, we need integration tests to verify that values are encrypted and decrypted seamlessly. Here is an example:
The complete code for this article can be found in this GitHub repository. You will notice that I have continuous integration set up and I am running the tests. To achieve this, I used two ingredients:
- I run a Postress database on Docker via GitHub actions. This is done in the “services” section of this file: https://github.com/algorythmist/aws-encryption-sdk-spring/blob/main/.github/workflows/maven.yml
- I am setting up the AWS environment variables as secrets on GitHub. This article describes how to do this.
Localstack
Finally, as I promised, I will show you how to set up KMS on AWS localstack so that you don’t need to connect to the actual Amazon servers.
Step 1: Install localstack
Before installing localstack, you need to install Docker and Python. Once python is installed, you can optionally create a venv and then install the following libraries:
pip install awscli
pip install awscli-local
pip install localstack
Detailed instructions can be found here.
Step 2: Define a docker-compose.yml file
version: '3.9'
services:
postgres:
container_name: postgres-aws-demo
hostname: postgres
image: postgres:14.1-alpine
ports:
- "5432:5432"
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
localstack:
container_name: localstack
image: localstack/localstack:latest
environment:
- AWS_DEFAULT_REGION=us-east-1
- EDGE_PORT=4566
- SERVICES=lambda,s3,kms
ports:
- "4566:4566" # The Edge port can multiplex all services
volumes:
- "${TEMPDIR:-/tmp/localstack}:/tmp/localstack"
- "/var/run/docker.sock:/var/run/docker.sock"
Step 3: Create a secret key
awslocal kms create-key
To look at the secret key identities use:
awslocal kms list-keys
Detailed instructions can be found here.
Step 4: Connect to localstack
As I mentioned before, the program as it is will attempt to connect to an AWS server and will look in the environment variables for AWS_ACCESS_KEY and AWS_SECRET_ACCESS_KEY.
Since we will be running KMS locally, we need to connect to a local server. To achieve this, I introduce the following modifications to the Spring configuration:
Note that I am now using the configuration parameter “aws.localstack” to control if I am connecting to localstack or the actual AWS. If it is set to true, I redirect the endpoint to “http://127.0.0.1:4566" and set the credentials explicitly. It does not matter what the credentials are, since they are ignored, but they must be set to something, so I set them to empty strings.
Level Up Coding
Thanks for being a part of our community! Before you go:
- 👏 Clap for the story and follow the author 👉
- 📰 View more content in the Level Up Coding publication
- 🔔 Follow us: Twitter | LinkedIn | Newsletter
🚀👉 Join the Level Up talent collective and find an amazing job
AWS Encryption SDK was originally published in Level Up Coding on Medium, where people are continuing the conversation by highlighting and responding to this story.
This content originally appeared on Level Up Coding - Medium and was authored by Dimitri Papaioannou
Dimitri Papaioannou | Sciencx (2022-11-20T21:30:34+00:00) AWS Encryption SDK. Retrieved from https://www.scien.cx/2022/11/20/aws-encryption-sdk/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.