AWS Data migration service (DMS) configuration considerations

A data migration to AWS involves leveraging various AWS services tailored to the type of data being migrated:

• Relational Database Migration
  Use AWS Database Migration Service (DMS) for migrating relational databases.
  Additionally, native database tools such as Oracle GoldenGate, SQL Server Replication, bulk load, etc., can be considered based on the source and target environments.

• File-Based Migration
  Utilize services like AWS Transfer Family and AWS DataSync etc for migrating file-based data efficiently and securely.

• Real-Time Data Migration
  For streaming or real-time data, consider Amazon Kinesis or Amazon MSK (Managed Streaming for Apache Kafka) to ensure low-latency and scalable data movement.

In this blog, we will explore AWS Database Migration Service (DMS)—its core capabilities, configuration options, and key considerations around cost and optimization. Whether one is planning a large-scale migration or a targeted database shift, understanding how to configure DMS effectively and manage associated costs is essential for a successful cloud transition.

AWS DMS:

AWS Database Migration Service is a cloud service to migrate relational databases, NoSQL databases, data warehouses and all other type of data stores into AWS Cloud or between cloud and on-premises setups efficiently and securely. DMS supports several types of source and target databases such as Oracle, MS SQL Server, MySQL, Postgres SQL, Amazon Aurora, AWS RDS, Redshift and S3 etc.

The service runs on an Amazon EC2 instance (Replication instance), which performs the following tasks:

1. Reads data from the source database.
2. Transforms/formats the data as needed.
3. Loads the data into the target database.

Components to consider for cost factors for AWS DMS are as follows:

• Instance class:
  This is the type of instance class considered for the replication instance for the actual data migration. AWS DMS supports T2, T3, C4, C5, C6i, R4, R5 and R6i instance classes.

• Storage cost:
  Each of these instances will have 50/100GBs or more of the storage attached for data cache, replication logs with $0.115 GB/month for single AZ instance.

• Multi-AZ:
  Single-AZ or Multi-AZ for high availability of the above configurations would increase the cost of above configuration accordingly.

• Data Transfer:
  A standard data transfer cost for any data out of the region or the AWS account.

Database migrations and Change Data Capture (CDC) are inherently complex processes influenced by multiple factors. To ensure a successful and optimized migration using AWS Database Migration Service (DMS), it is essential to carefully evaluate and configure the service based on the specific data migration requirements:

• Source database engine and versions.

• Target database engine and versions.

• Is target database relational or non-relational database.

• Homogeneous or heterogeneous data migration for SCD tool considerations accordingly.

• Requirement of one-time data migration or continuous data replication.

• Selected tables or entire database.

• Number of tables, columns, transformations required.

• LOB columns to consider for the data migration.

• In case of CDC, acceptable amount of replication latency.

• Rate of change of data at source during peak hours.

• Source and target database utilization during peak hours.

• Transaction log size.

• Network connectivity between source and target.

• Network latency between source to replication instance and replication instance and target.

• Additional source database engine configurations such as storage on ASM, RAC implementation for Oracle.

• Execute AWS provided scripts on the source database to capture configuration details.

According to above points one need to consider and configure:

• Replication instance sizing.

• Latest version of the AWS DMS.

• Sufficient storage on the replication instance for transaction logs and buffering of the data during peak hours.

• Limitations for data migration for AWS DMS for each of the source and target databases.

• Multi-AZ configurations: Lower environments like Dev, SIT and UAT can be configured in Single-AZ and Prod environment with multi-AZ for high availability.

One can test the DMS implementation by starting with subset of the data and adding tables gradually and accordingly changing the DMS configurations. Document all these details.

Conclusion:
This is a blog to consider all the points for before performing the relational databases migration using AWS DMS. This will help to consider all the factors affecting the database migrations.