DEV Community

AWS Community Builders profile image Vadym Kazulkin
Vadym Kazulkin for AWS Community Builders

Posted on

How to set up Amazon RDS Proxy for Amazon Aurora (Serverless) database cluster and connect AWS Lambda function to it

#aws #serverless #postgres #java

Introduction

In this article we'd like to explore how to connect Lamdba function to the Aurora database through the RDS Proxy using Java JDBC capabilities. You can explore RDS Proxy capabilities in the Using Amazon RDS Proxy article. We will use Aurora Serverless v2 PostgreSQL database in this article but the same will also work Amazon Aurora PostgreSQL-Compatible Edition, Amazon Aurora MySQL-Compatible Edition, Amazon RDS for PostgreSQL, Amazon RDS for MySQL, Amazon RDS for MariaDB, and Amazon RDS for SQL Server as well. And it's easy to re-write Lambda function using different programming language as the basic concepts remain the same.

Solution

What we'd like to achieve is described in the following architecture diagram:

We'll focus on the part where Lambda connects through RDS Proxy to Aurora database. As the code sample we will use what we have created as part of the Data API for Amazon Aurora Serverless v2 with AWS SDK for Java series, and the project can be found on my GitHub repository. The relevant part of the Infrastructure as a Code can be found in this AWS SAM template.

First of all, we need to create VPC to put Lambda, RDS Proxy and Aurora Cluster to, so that Lambda is able to talk to RDS Proxy which in turn connects to Aurora Cluster. For that, please define your own VPC Id and Subnet list in the "Parameter" section of the SAM template like this :

 VpcId: Type: String Default: vpc-950cd6fd Description: VpcId of your existing Virtual Private Cloud (VPC) Subnets: Type: CommaDelimitedList Default: subnet-0787be4d, subnet-88dc46e0 Description: The list of SubnetIds, for at least two Availability Zones in the region in your Virtual Private Cloud (VPC)
Enter fullscreen mode Exit fullscreen mode

Then we need to define Lambda Security Group which references the VPC Id:

 LambdaSecurityGroup: Type: AWS::EC2::SecurityGroup Properties: GroupDescription: SecurityGroup for Serverless Functions VpcId: Ref: VpcId
Enter fullscreen mode Exit fullscreen mode

and then VPC Security Group itself:

 VPCSecurityGroup: Type: AWS::EC2::SecurityGroup Properties: GroupDescription: Security group for RDS DB Instance. VpcId: Ref: VpcId SecurityGroupEgress: - CidrIp: '0.0.0.0/0' Description: lambda RDS access over 5432 FromPort: 5432 IpProtocol: TCP ToPort: 5432 SecurityGroupIngress: - IpProtocol: tcp FromPort: '5432' ToPort: '5432' SourceSecurityGroupId: Ref: LambdaSecurityGroup
Enter fullscreen mode Exit fullscreen mode

VPC Security Group defines Security Group engress and igress roules which enable Lambda to talk to RDS Proxy via the port number 5432 (default port for PostgreSQL) and references the defined Lambda Security Group.

Now let's create Aurora Serverless v2 PostgreSQL database cluster. But before the let's create SecretsManager to store the database user and password:

 DBSecret: Type: AWS::SecretsManager::Secret Properties: Name: !Ref UserSecret Description: RDS database auto-generated user password GenerateSecretString: SecretStringTemplate: !Sub '{"username": "${DBMasterUserName}"}' GenerateStringKey: "password" PasswordLength: 30 ExcludeCharacters: '"@/\'
Enter fullscreen mode Exit fullscreen mode

Then we need to define DB Subnet Group:

 DBSubnetGroup: Type: AWS::RDS::DBSubnetGroup Properties: DBSubnetGroupDescription: Subnets available for the RDS DB Instance SubnetIds: Ref: Subnets
Enter fullscreen mode Exit fullscreen mode

and then create the Aurora Cluster with the engine "aurora-postgresql" itself:

 AuroraServerlessV2Cluster: Type: 'AWS::RDS::DBCluster' DeletionPolicy: Delete Properties: DBClusterIdentifier: !Ref DBClusterName Engine: aurora-postgresql Port: 5432 EnableHttpEndpoint: true MasterUsername: !Join ['', ['{{resolve:secretsmanager:', !Ref DBSecret, ':SecretString:username}}' ]] MasterUserPassword: !Join ['', ['{{resolve:secretsmanager:', !Ref DBSecret, ':SecretString:password}}' ]] DatabaseName: !Ref DatabaseName ServerlessV2ScalingConfiguration: MinCapacity: 0.5 MaxCapacity: 1 DBSubnetGroupName: Ref: DBSubnetGroup VpcSecurityGroupIds: - !Ref VPCSecurityGroup
Enter fullscreen mode Exit fullscreen mode

In the last lines we reference already created DB Subnet Group and Vpc Security Group Ids. With ServerlessV2ScalingConfiguration we define the scaling behaviour of our Aurora Serverless V2 Cluster.

After it we need to create database instance of the Aurora (Serverless v2) cluster :

 AuroraServerlessV2Instance: Type: 'AWS::RDS::DBInstance' Properties: Engine: aurora-postgresql DBInstanceClass: db.serverless DBClusterIdentifier: !Ref AuroraServerlessV2Cluster MonitoringInterval: 1 MonitoringRoleArn: !GetAtt EnhancedMonitoringRole.Arn PubliclyAccessible: false EnablePerformanceInsights: true PerformanceInsightsRetentionPeriod: 7
Enter fullscreen mode Exit fullscreen mode

Now let's take care of the creation of the RDS Proxy itself :

 RDSProxy: Type: AWS::RDS::DBProxy Properties: Auth: - { AuthScheme: SECRETS, SecretArn: !Ref DBSecret} DBProxyName: 'rds-proxy' RoleArn: !GetAtt RDSProxyRole.Arn EngineFamily: 'POSTGRESQL' IdleClientTimeout: 120 RequireTLS: true DebugLogging: false VpcSecurityGroupIds: - !Ref VPCSecurityGroup VpcSubnetIds: !Ref Subnets
Enter fullscreen mode Exit fullscreen mode

We use EngineFamily 'POSTGRESQL' and AuthScheme connected to the created SecretsManager secret for authentication in our scenario. There are also other authentication possibilities like IAM authentication offered. We also connect RDS Proxy to the already created Vpc Security Group and Vpc Subnet Id.

We also need to create RDS Proxy Role to allow RDS Proxy to fetch secrets from the SecretsManager:

 RDSProxyRole: Type: AWS::IAM::Role Properties: Path: / AssumeRolePolicyDocument: Version: '2012-10-17' Statement: - Action: [ 'sts:AssumeRole' ] Effect: Allow Principal: Service: [ rds.amazonaws.com ] Policies: - PolicyName: DBProxyPolicy PolicyDocument: Version: '2012-10-17' Statement: - Action: - secretsmanager:GetSecretValue Effect: Allow Resource: - !Ref DBSecret
Enter fullscreen mode Exit fullscreen mode

As the final step we need to create DB Proxy Target Group:

 ProxyTargetGroup: Type: AWS::RDS::DBProxyTargetGroup Properties: DBProxyName: !Ref RDSProxy DBClusterIdentifiers: [ !Ref AuroraServerlessV2Cluster] TargetGroupName: default ConnectionPoolConfigurationInfo: MaxConnectionsPercent: 5 MaxIdleConnectionsPercent: 4 ConnectionBorrowTimeout: 120
Enter fullscreen mode Exit fullscreen mode

DB Proxy Target Group connects RDS Proxy with the Aurora V2 Cluster and defines Connection Pool Configuration. Please refer to Configure Connection Settings documentation for the extended explanation of this configuration.

Now let's connect Lambda function "GetProductByIdViaAuroraServerlessV2WithRDSProxy" to the RDS Proxy.
The relevant part is this one:

 Policies: - Statement: - Sid: AllowDbConnect Effect: Allow Action: - rds-db:connect Resource: - !Sub arn:aws:rds-db:${AWS::Region}:${AWS::AccountId}:dbuser:!Select [6, !Split [":", !GetAtt RDSProxy.DBProxyArn]]/* - VPCAccessPolicy: {} VpcConfig: SecurityGroupIds: - Fn::GetAtt: LambdaSecurityGroup.GroupId SubnetIds: !Ref Subnets
Enter fullscreen mode Exit fullscreen mode

In the VPCAccessPolicy part we reference already created Security Groups and Subnets. In the Policies part we allow Lambda to connect to our RDS Proxy instance by defining the exact resource ARN by knowing how the resource schema name works but extracting the appropriate RDSProxy ARN part from it. For the exact explanation please visit Creating and using an IAM policy for IAM database access.

The word of caution: only for the demonstration purpose I passed the database name and password as the Lambda environment variables to connect to RDS Proxy which introduces the security risk.

 Environment: Variables: DB_USER_PASSWORD: !Join ['', ['{{resolve:secretsmanager:', !Ref DBSecret, ':SecretString:password}}' ]] DB_USER_NAME: !Join ['', ['{{resolve:secretsmanager:', !Ref DBSecret, ':SecretString:username}}' ]] RDS_PROXY_ENDPOINT: !GetAtt RDSProxy.Endpoint
Enter fullscreen mode Exit fullscreen mode

The proper solution is to use the stored database name and password in Amazon Secret Manager and then retrieve the in the Lambda function itself.

That it on the Infrastructure as a Code level.

Everything else will be done in the Lambda handler itself to connect to the RDS Proxy. In my case I wrote GetProductByIdViaAuroraServerlessV2RDSProxyHandler in Java and used JDBC for the connection with the PostgreSQL database driver in the pom.xml like this.

 <dependency> <groupId>org.postgresql</groupId> <artifactId>postgresql</artifactId> <version>42.5.4</version> </dependency>
Enter fullscreen mode Exit fullscreen mode

Each programming languge in which Lambda function can be implemented offers its own capabilities to connect to the database.

Conclusion

In this article we explored how to connect Lamdba function to the Aurora database through the RDS Proxy and demonstrated the example of the corresponding Infrastructure as a Code part and the Lambda function written in Java which uses JDBC to connect to RDS Proxy.

Top comments (1)

Collapse
 
sundus profile image
Sundus Hussain •

nice