AWS Secrets Manager, RDS and Spring Boot example using Docker

AWS Secrets Manager helps you protect secrets needed to access your applications, services, and IT resources.

You must not store sensitive data such as database credentials in your repository (Git). AWS secrete manager provides APIs to retrieve application secrets when deploying the applications. It also has built-in integration for Amazon RDS, Amazon Redshift, and Amazon DocumentDB.

First of all, we created a Database in AWS RDS and stored the database credentials in the AWS Secrets Manager.

AWS secret manager

We will access this secret while setting up the Spring data sources. Check the below code segment.


In the above code, we have retrieved the database secrets and configured HikarCP to create a data source.

According to Spring Cloud AWS documentation, to configure Parameter Store and Secrets Manager by overriding, we need to define our own Spring Cloud bootstrap configuration class with a bean of type AWSSimpleSystemsManagement. Instead of doing this, I created a Bean which connects only to AWS Secrets manager to retrieve the sensitive data.

In this way, I can create an AWSSecretsManager with the region.
There is another library called aws-secretsmanager-jdbc which supports AWS RDS integration, but I encountered several difficulties when working with it. I have explained this in a later part.

Setting up a tunnel to the AWS RDS for Local Development

In my project, I needed to connect to the secure development database in AWS RDS. This database is running in a private subnet in a VPC. It cannot be accessed directly from my local development environment.

To connect to the database, I used a mechanism called local port forwarding.
Read more: https://www.ssh.com/ssh/tunneling/example

The command I used is this :
ssh -i testApiEc2.pem -N -L 3306:dev-xxxxxx-database.cluster-xxxxxxxx.ap-southeast-1.rds.amazonaws.com:3306 ec2-user@54.254.xxx.xxx

Running the project with Eclipse

When running the project with eclipse, you have to pass the database host with the VM arguments because the host is a configurable value. AWS region is in the code itself and Aws credentials details are retrieved from the .aws folder.

eclipse_vm_arguements

Running the project with Docker

When running the project with Docker, we need to pass the credentials with the database host.

Using AWS Secrets Manager JDBC Library (Another way to do this)

https://github.com/aws/aws-secretsmanager-jdbc

This is one of the approaches I used before. With this library, we only need to add the proper configuration to the "application.property" file. Retrieving the secret and connecting to the AWS RDS will be handled by this library.

spring.datasource.url=jdbc-secretsmanager:mysql://localhost:3306/dev_morning

spring.datasource.username=/secret/morning-api-rds-dev

spring.datasource.driver-class-name=com.amazonaws.secretsmanager.sql.AWSSecretsManagerMySQLDriver

spring.datasource.testOnBorrow=true

spring.datasource.validationQuery=SELECT 1

spring.datasource.hikari.maximumPoolSize=2

The issue with this approach was, when I tried to run this project with Docker, it was not able to detect the AWS region inside docker.
This library retrieves the AWS region and credential details from the .aws/config and .aws/credentials files on your computer and my docker container did not contain .aws/config and .aws/credentials files.

Adding the region to application.properties class did not configure the AWS region properly.

Check this thread: https://github.com/spring-cloud/spring-cloud-aws/issues/477

Code for the project: https://github.com/h-hub/Spring_Boot_AWS_SecreteManager_RDS

References

• https://cloud.spring.io/spring-cloud-aws/spring-cloud-aws.html#_mixing_both_security_configurations
• https://cloud.spring.io/spring-cloud-commons/multi/multi__spring_cloud_context_application_context_services.html
• https://www.ssh.com/ssh/tunneling/example
• https://aws.amazon.com/secrets-manager/
• https://docs.docker.com/compose/