DEV Community

Cover image for How to manage multiple environments with Terraform using workspaces?
Citrux Digital profile image munikeraragon
munikeraragon for Citrux Digital

Posted on • Originally published at citruxdigital.com

How to manage multiple environments with Terraform using workspaces?

When developing enterprise applications, it's crucial to manage different environments like development, staging, and production. This is considered a best practice in the industry, and it's something that big tech companies rely on to maintain stability and ensure smooth deployments. Terraform Workspaces help you do this by keeping these environments separate without needing to duplicate your configuration files. This approach allows you to test changes safely before pushing them to production.

For example, if you're working on a large-scale application, testing new features in a staging environment before they go live is essential. Terraform Workspaces make this possible by creating isolated copies of your infrastructure. Once you're confident the changes work as expected, you can move them to production with less risk, just like the big tech companies do.

In this blog, I'll show you how to set up and use Terraform Workspaces to manage multiple environments effectively. We’ll use a practical example based on the repository terraform-api-workspaces.

Why Use Workspaces for Enterprise Applications?

Using Workspaces is not just a good idea—it’s an industry-standard practice. It helps you manage different environments without mixing them up, keeps things organized, reduces errors, and ensures you can test thoroughly before making any changes live. This is how major tech companies maintain reliable and scalable infrastructure.

Get Started with Workspaces

Follow the steps below to learn how to manage multiple environments with Terraform Workspaces. By the end of this guide, you'll be able to keep your environments separate and your infrastructure stable, following the same practices used by industry leaders.

Step-by-step guide: Managing Multiple Environments with Terraform Workspaces

Before starting, check out our previous post: Step-by-Step Guide: Deploying a REST API in AWS with Terrafom. It covers the basics you’ll need to follow along.

First step: Fork repository

First, fork the repository rest-api-aws-terraform This repo has the Terraform configurations we'll use to demonstrate how Workspaces work.

Repository to be forked

Second step: initialize Terraform

Execute the command bellow:

$ make init
Enter fullscreen mode Exit fullscreen mode

That is for initialize Terraform

Third Step: Create the Workspaces

In the directory /terraform

At this point you can see what are the workspace created with this command terraform workspace list

$ terraform workspace list * default
Enter fullscreen mode Exit fullscreen mode

Create the testing workspace with the command terraform workspace new testing

$ terraform workspace new testing Created and switched to workspace 'testing'! You're now on a new, empty workspace. Workspaces isolate their state, so if you run 'terraform plan' Terraform will not see any existing state for this configuration.
Enter fullscreen mode Exit fullscreen mode

Create the development workspace with the command terraform workspace new development

$ terraform workspace new development Created and switched to workspace 'development'! You're now on a new, empty workspace. Workspaces isolate their state, so if you run 'terraform plan' Terraform will not see any existing state for this configuration.
Enter fullscreen mode Exit fullscreen mode

Create the production workspace with the command terraform workspace new production

$ terraform workspace new production Created and switched to workspace 'production'! You're now on a new, empty workspace. Workspaces isolate their state, so if you run 'terraform plan' Terraform will not see any existing state for this configuration.
Enter fullscreen mode Exit fullscreen mode

When a new workspace is created, it automatically switches to that environment. You can switch to another workspace with terraform workspace select <workspace_name>

$ terraform workspace select development Switched to workspace 'development'.
Enter fullscreen mode Exit fullscreen mode

You should have a list like this:

$ terraform workspace list default * development production testing
Enter fullscreen mode Exit fullscreen mode

Fourth step: Config the modules to work with workspaces

In the file /terraform/variables config the variables to be use with workspaces

... # Variable for infrastructure environment variable 'environment' { description = 'Environment to deploy resources' # Description of the environment variable type = map(string) # Data type of the variable default = { production = 'infra-prod' development = 'infra-dev' testing = 'infra-test' } }
Enter fullscreen mode Exit fullscreen mode

Modific the file /terraform/main.tf to use the new variables

# VPC Module module 'vpc' { source = './modules/vpc' vpc_environment = var.environment[terraform.workspace] # Add this environment for VPC deployment } # API Gateway Module module 'api-gateway' { source = './modules/api-gateway' users_lambda_invoke_arn = module.lambda.user_lambda_arn aws_region = var.aws_region api_gateway_env = var.environment[terraform.workspace] # Add this environment for API Gateway deployment } # Lambda Module module 'lambda' { source = './modules/lambda' subnets_ids = module.vpc.subnets_ids lambda_vpc_id = module.vpc.lambda_vpc_id lambda_env = var.environment[terraform.workspace] # Add this environment for Lambda deployment }
Enter fullscreen mode Exit fullscreen mode

Module api-gateway: In the module api-gateway (/terraform/modules/api-gateway) in the file variables

Add the new variable api_gateway_env

... variable 'api_gateway_env' { description = 'Environment to deploy API Gateway resources' type = string }
Enter fullscreen mode Exit fullscreen mode

Change the name of the resources in the main file of the module concatenating the new variable + the resource name

# Create an IAM role for API Gateway to push logs to CloudWatch resource 'aws_iam_role' 'api_gateway_cloudwatch_role' { name = '${var.api_gateway_env}-api-gateway-cloudwatch-role' # Name of the IAM role ... } ... # Create a CloudWatch log group for API Gateway logs resource 'aws_cloudwatch_log_group' 'api_logs' { name = '/aws/api_gateway/${var.api_gateway_env}-example-api' # Name of the log group } # Create an API Gateway REST API resource 'aws_api_gateway_rest_api' 'api' { name = '${var.api_gateway_env}-example-api' # Name of the API Gateway ... } ... # Create a stage for the API Gateway REST API resource 'aws_api_gateway_stage' 'api_gateway_stage' { ... stage_name = '${var.api_gateway_env}-stage' # Name of the stage ... } # Define the Cognito User Pool Authorizer for the API Gateway resource 'aws_api_gateway_authorizer' 'cognito_authorizer' { name = '${var.api_gateway_env}-cognito-authorizer' ... } # Define a Cognito User Pool resource 'aws_cognito_user_pool' 'main' { name = '${var.api_gateway_env}-user-pool' ... } # Define a Cognito User Pool Domain resource 'aws_cognito_user_pool_domain' 'main' { domain = '${var.api_gateway_env}-pool-domain' # Replace with your desired domain prefix ... } # Define a Cognito User Pool Client resource 'aws_cognito_user_pool_client' 'main' { name = '${var.api_gateway_env}-user-pool-client' ... }
Enter fullscreen mode Exit fullscreen mode

Module vpc: In the module vpc (/terraform/modules/vpc) in the file variables

Add the new variable vpc_environment

variable 'vpc_environment' { description = 'Environment to deploy VPC resources' type = string }
Enter fullscreen mode Exit fullscreen mode

Change the name of the resources in the main file of the module concatenating the new variable + the resource name

// Create a VPC resource 'aws_vpc' 'lambda_vpc' { ... tags = { Name = '${var.vpc_environment}_lambda_vpc' Environment = var.vpc_environment } } // Create a public subnet resource 'aws_subnet' 'public_subnet' { ... tags = { Name = '${var.vpc_environment}_public_subnet' Environment = var.vpc_environment } } // Create the first private subnet resource 'aws_subnet' 'lambda_subnet_a' { ... tags = { Name = '${var.vpc_environment}_lambda_subnet_a' Environment = var.vpc_environment } } // Create the second private subnet resource 'aws_subnet' 'lambda_subnet_b' { ... tags = { Name = '${var.vpc_environment}_lambda_subnet_b' Environment = var.vpc_environment } } ... // Create a NAT gateway in the public subnet resource 'aws_nat_gateway' 'nat_gw' { ... tags = { Name = '${var.vpc_environment}_nat_gw' Environment = var.vpc_environment } } // Create an Internet Gateway for the VPC resource 'aws_internet_gateway' 'igw' { ... tags = { Name = '${var.vpc_environment}_igw' Environment = var.vpc_environment } } // Define a route table for the public subnet resource 'aws_route_table' 'public_rt' { ... tags = { Name = '${var.vpc_environment}_public_rt' Environment = var.vpc_environment } } // Define a route table for the private subnets resource 'aws_route_table' 'private_rt' { ... tags = { Name = '${var.vpc_environment}_private_rt' Environment = var.vpc_environment } } ... // Define a default security group for instances resource 'aws_security_group' 'primary_default' { name_prefix = '${var.vpc_environment}-default-' description = 'Default security group for all instances in ${aws_vpc.lambda_vpc.id}' ... tags = { Name = '${var.vpc_environment}_primary_default_sg' Environment = var.vpc_environment } }
Enter fullscreen mode Exit fullscreen mode

Module lambda: In the module lambda (/terraform/modules/lambda) in the file variables

Add the new variable lambda_env

variable 'lambda_env' { description = 'Environment to deploy Lambda resources' type = string }
Enter fullscreen mode Exit fullscreen mode

Change the name of the resources in the main file of the module concatenating the new variable + the resource name

# Create the Lambda function for users resource 'aws_lambda_function' 'users' { function_name = '${var.lambda_env}-usersExampleLambda' # Name of the Lambda function ... } ... # Create an IAM role that the Lambda function will assume resource 'aws_iam_role' 'lambda_exec' { name = '${var.lambda_env}-lambda_exec_role' # Name of the IAM role ... } # Create an IAM policy that allows Lambda to log to CloudWatch Logs resource 'aws_iam_policy' 'lambda_logging' { name = '${var.lambda_env}-LambdaLogging' # Name of the IAM polic ... } ... # Create an IAM policy that allows Lambda to manage ENIs for VPC access resource 'aws_iam_policy' 'lambda_vpc_access' { name = '${var.lambda_env}-LambdaVPCAccess' # Name of the IAM policy ... } ... # Create a security group for the Lambda function resource 'aws_security_group' 'lambda_sg' { ... name = '${var.lambda_env}-lambda_sg' # Name of the security group ... }
Enter fullscreen mode Exit fullscreen mode

Deploying infrastructure

With these changes now we can deploy the infrastructure for each workspace, from the root of the project

  • run make plan for see the resource that going to be created
  • run make apply for create the resources on AWS

Change workspace with the command terraform workspace select <workspace_name> and do the deploy of AWS on all the workspaces

Check the AWS console to verify the resources created

  • Resources created on VPC

Resources created on VPC: test dev prod

  • Resources created on api-gateway

Resources created on api gateway: dev, prod, test

  • Resources created on lambda

Resources created on lambda: test, dev, prod

How this works?

Terraform Workspaces utilize environment-specific variables to maintain isolation between different stages of your deployment, such as development, testing, and production. When you switch between workspaces, Terraform automatically uses the corresponding environment variables, ensuring that each workspace operates with its unique set of configurations.

For example, if you are working in the development workspace, Terraform will use environment variables specific to development, such as 'infra-dev'. These variables are defined in the /terraform/variables.tf file and are used throughout your Terraform configurations to differentiate resources between environments. This ensures that your development infrastructure does not interfere with your production setup.

By leveraging these environment variables, Terraform enables seamless transitions between workspaces, making it easier to manage multiple environments and maintain consistency across your infrastructure.

Conclusion

Managing multiple environments with Terraform workspaces is a powerful practice that allows you to keep your infrastructure consistent and organized across different stages of development, testing, and production. By using workspaces, you can ensure that changes are thoroughly tested before being applied to your production environment, reducing the risk of disruptions and errors.

This guide walked you through the process of setting up Terraform workspaces, configuring variables, and deploying infrastructure in each workspace. By following these steps, you can maintain clean and efficient infrastructure-as-code configurations that adapt to your project's needs.

✨Connect with us today and take your software development to the next level. See our services for more details.

Top comments (0)