DEV Community

Cover image for Caching DynamoDB Results with Redis using AWS Lambda + API Gateway (with Terraform)
Kachi
Kachi

Posted on

Caching DynamoDB Results with Redis using AWS Lambda + API Gateway (with Terraform)

#redis #apigateway #lambda #terraform

Caching is one of the most effective ways to improve application performance while reducing costs. In this guide, I'll show you how to implement a cache-aside pattern using DynamoDB, ElastiCache Redis, AWS Lambda, and API Gateway - all provisioned with Terraform.

📘 Part 1: Infrastructure Setup

1. DynamoDB Table

First, let's create a Products table in DynamoDB:

resource "aws_dynamodb_table" "products" { name = "Products" billing_mode = "PAY_PER_REQUEST" hash_key = "productId" attribute { name = "productId" type = "S" } }
Enter fullscreen mode Exit fullscreen mode

2. ElastiCache Redis Cluster

We'll deploy Redis inside a VPC for better security and performance:

resource "aws_elasticache_cluster" "products_cache" { cluster_id = "products-cache" engine = "redis" node_type = "cache.t3.micro" num_cache_nodes = 1 parameter_group_name = "default.redis6.x" engine_version = "6.x" port = 6379 security_group_ids = [aws_security_group.redis.id] subnet_group_name = aws_elasticache_subnet_group.redis.name } resource "aws_elasticache_subnet_group" "redis" { name = "redis-subnet-group" subnet_ids = [aws_subnet.private_a.id, aws_subnet.private_b.id] }
Enter fullscreen mode Exit fullscreen mode

3. Networking Configuration

Proper VPC setup is crucial:

resource "aws_vpc" "main" { cidr_block = "10.0.0.0/16" } resource "aws_subnet" "private_a" { vpc_id = aws_vpc.main.id cidr_block = "10.0.1.0/24" availability_zone = "us-east-1a" } resource "aws_security_group" "redis" { name = "redis-sg" description = "Allow access to Redis" vpc_id = aws_vpc.main.id ingress { from_port = 6379 to_port = 6379 protocol = "tcp" security_groups = [aws_security_group.lambda.id] } }
Enter fullscreen mode Exit fullscreen mode

📘 Part 2: Lambda Function Logic

Here's our Python Lambda function implementing the cache-aside pattern:

import os import json import boto3 import redis from datetime import datetime # Initialize clients dynamodb = boto3.resource('dynamodb') products_table = dynamodb.Table('Products') # Redis connection redis_client = redis.Redis( host=os.environ['REDIS_HOST'], port=6379, decode_responses=True ) def lambda_handler(event, context): product_id = event['pathParameters']['productId'] cache_key = f"product:{product_id}" # Try to get from Redis first  cached_product = redis_client.get(cache_key) if cached_product: print("Cache hit!") return { 'statusCode': 200, 'body': cached_product } print("Cache miss - fetching from DynamoDB") # Get from DynamoDB  response = products_table.get_item(Key={'productId': product_id}) if 'Item' not in response: return {'statusCode': 404, 'body': 'Product not found'} product = response['Item'] product_json = json.dumps(product) # Cache with 5 minute TTL  redis_client.setex(cache_key, 300, product_json) return { 'statusCode': 200, 'body': product_json }
Enter fullscreen mode Exit fullscreen mode

📘 Part 3: API Gateway Configuration

Let's expose our Lambda through API Gateway:

resource "aws_api_gateway_rest_api" "products_api" { name = "products-api" } resource "aws_api_gateway_resource" "product" { rest_api_id = aws_api_gateway_rest_api.products_api.id parent_id = aws_api_gateway_rest_api.products_api.root_resource_id path_part = "product" } resource "aws_api_gateway_resource" "product_id" { rest_api_id = aws_api_gateway_rest_api.products_api.id parent_id = aws_api_gateway_resource.product.id path_part = "{productId}" } resource "aws_api_gateway_method" "get_product" { rest_api_id = aws_api_gateway_rest_api.products_api.id resource_id = aws_api_gateway_resource.product_id.id http_method = "GET" authorization = "NONE" } resource "aws_api_gateway_integration" "lambda" { rest_api_id = aws_api_gateway_rest_api.products_api.id resource_id = aws_api_gateway_resource.product_id.id http_method = aws_api_gateway_method.get_product.http_method integration_http_method = "POST" type = "AWS_PROXY" uri = aws_lambda_function.get_product.invoke_arn }
Enter fullscreen mode Exit fullscreen mode

📘 Part 4: Monitoring and Optimization

1. Adding TTLs

We already implemented TTLs in our Lambda function (setex with 300 seconds), but let's add CloudWatch metrics to track cache performance:

from aws_lambda_powertools import Metrics metrics = Metrics() def lambda_handler(event, context): # ... existing code ...  if cached_product: metrics.add_metric(name="CacheHits", unit="Count", value=1) # ... return cached product ...  else: metrics.add_metric(name="CacheMisses", unit="Count", value=1) # ... fetch from DynamoDB ...
Enter fullscreen mode Exit fullscreen mode

2. Terraform for Monitoring

Add CloudWatch alarms and dashboards:

resource "aws_cloudwatch_dashboard" "cache" { dashboard_name = "cache-performance" dashboard_body = jsonencode({ widgets = [ { type = "metric" x = 0 y = 0 width = 12 height = 6 properties = { metrics = [ ["AWS/Lambda", "CacheHits", "FunctionName", aws_lambda_function.get_product.function_name], ["AWS/Lambda", "CacheMisses", "FunctionName", aws_lambda_function.get_product.function_name] ] period = 300 stat = "Sum" region = "us-east-1" title = "Cache Performance" } } ] }) }
Enter fullscreen mode Exit fullscreen mode

3. Complete Terraform Workflow

For a production setup, add a CI/CD pipeline:

resource "aws_codepipeline" "deploy_pipeline" { name = "products-api-deployment" role_arn = aws_iam_role.codepipeline.arn artifact_store { location = aws_s3_bucket.artifacts.bucket type = "S3" } stage { name = "Source" action { name = "Source" category = "Source" owner = "ThirdParty" provider = "GitHub" version = "1" output_artifacts = ["source_output"] configuration = { Owner = "your-github-org" Repo = "products-api" Branch = "main" OAuthToken = var.github_token } } } stage { name = "Terraform" action { name = "Apply" category = "Build" owner = "AWS" provider = "CodeBuild" input_artifacts = ["source_output"] version = "1" configuration = { ProjectName = aws_codebuild_project.terraform.name } } } }
Enter fullscreen mode Exit fullscreen mode

Results and Observations

After implementing this architecture, you should see:

  • Average latency reduction from ~100ms (DynamoDB) to ~5ms (Redis) for cache hits
  • Reduced DynamoDB RCU consumption (and costs) by 80-90% for frequently accessed items
  • More consistent performance under load

Final Thoughts

This cache-aside pattern is just one of many ways to optimize DynamoDB performance. For production workloads, consider:

  • Adding write-through caching for data modifications
  • Implementing cache invalidation strategies
  • Monitoring Redis memory usage and eviction policies
  • Considering DAX for alternative DynamoDB caching

The complete Terraform code is available in this GitHub repo.

Have you implemented similar caching patterns? Share your experiences in the comments!

Top comments (0)