Data Transmission Service:What is Data Transmission Service (DTS)?

Video introduction

Core capabilities

DTS provides powerful core capabilities centered around data flow, covering a wide range of scenarios from data migration and real-time synchronization to incremental change tracking. These capabilities are categorized as follows:

Benefits

Compared with traditional data migration or synchronization tools, DTS offers more features, better performance, greater ease of use, and enhanced security and reliability. It simplifies complex data interaction tasks. The main benefits of DTS are as follows:

• High compatibility

  • Support for disparate data sources: DTS supports data transmission between various data sources, such as relational databases (such as MySQL, PostgreSQL, SQL Server, and Oracle), NoSQL databases (such as MongoDB and Redis), and data warehouses (such as AnalyticDB and MaxCompute). For more information, see Supported databases.

  • Seamless cross-environment connectivity: DTS supports data flow between on-premises databases in data centers, databases on Alibaba Cloud (such as ApsaraDB RDS and PolarDB), and databases on other cloud platforms. This enables data collaboration in hybrid cloud and multicloud architectures.

  • Support for cross-account collaboration: To meet complex enterprise collaboration needs, DTS supports configuring tasks across Alibaba Cloud accounts. You can simply configure RAM authorization to securely share data between different accounts. This enables unified data management and flow within a corporate group or across departments.

  • Flexible network topologies: DTS supports multiple network connection types, such as the Internet, leased lines, VPN gateways, and smart gateways, to accommodate complex enterprise network environments.

• Rich features

  • Provides multiple transmission methods, such as data migration, data synchronization, and change tracking. For heterogeneous data source migration, DTS supports schema definition transformation.

  • The real-time synchronization feature supports one-way or two-way synchronization and can be used in various scenarios, such as geo-disaster recovery, active geo-redundancy, and real-time data warehousing.

• Easy to use

  DTS provides a visual management interface and a wizard-based link creation process, allowing you to easily create and manage transmission links. The console displays information such as the transmission status, progress, and performance of the link. In addition, DTS includes features such as resumable transmission from breakpoints, automatic repair, and automatic restart to effectively handle network or system exceptions.

• High performance

  DTS uses high-specification servers to ensure high transmission performance. For data migration, multiple underlying optimizations enable a peak performance of up to 70 MB/s for full data migration. For real-time synchronization, DTS can reduce concurrency granularity to the transaction level, achieving a peak performance of 30,000 records per second (RPS). DTS also supports concurrent, multi-threaded compressed transmission to reduce bandwidth usage.

• Secure and reliable

  The underlying architecture of DTS is built on a service cluster. If a single node fails, a fast switchover is performed to ensure high link stability. DTS provides 24/7 data accuracy verification. Secure transmission protocols, secure token authentication, and Secure Sockets Layer (SSL) encryption ensure data transmission reliability and security. In cross-border and cross-region synchronization scenarios, DTS uses leased lines for isolation to further enhance data security.

• Zero-downtime migration

  You can migrate data while the source database remains operational. This reduces application downtime to minutes.

Scenarios

• Zero-downtime database migration to the cloud

  When migrating an on-premises database from a data center (such as MySQL or Oracle) or a self-managed database on an ECS instance to an Alibaba Cloud database such as ApsaraDB RDS or PolarDB, you can use the data migration feature to avoid long service interruptions. This feature uses full migration and incremental synchronization to complete the data migration while the application continues to write data. Then, you can switch your application to the new cloud database.

• Build active geo-redundancy and disaster recovery architectures

  To achieve high availability and geo-disaster recovery for core business systems, you can use the data synchronization feature to establish real-time two-way synchronization between database instances in different regions. This feature ensures that data changes on either end are synchronized to the other in real time. If a failure occurs in one region, service traffic can be quickly switched to the other data center to ensure business continuity.

• Real-time data warehouse ETL and cache updates

  When business data changes and you need to update a downstream analytical data warehouse (such as AnalyticDB or ClickHouse) or a Redis cache in real time, you can use the change tracking feature. DTS captures incremental logs (such as binary logging or write-ahead logging (WAL)) from the source database and converts them into a structured JSON data stream. Your application can consume this data stream to write to the data warehouse or directly update the cache, which achieves real-time decoupling of the data link.

• Real-time data cleansing and formatting

  When synchronizing data from a production database to an analytical database or a test environment, you often need to process the data. For example, you might need to mask sensitive information such as customer phone numbers or ID cards, or merge the first and last name fields from the source database into a single full name field in the destination database. You can use the data transformation feature to perform these transformations in real time during data transmission. This eliminates the need for custom development on the destination side, simplifying the data link and ensuring data security.

• Data consistency verification

  After a major database migration or during long-term geo-disaster recovery synchronization, you can use the data verification feature to ensure data consistency. This feature automatically compares all data or a specified range of data and generates a detailed verification report. The report identifies which tables and rows have data discrepancies. This provides data-level confidence for switching services after migration and offers a monitoring tool for the health of long-term synchronization links to expose potential data inconsistencies.

Get started with DTS

1. Identify the product feature that suits your business scenario.

2. Check if your database is supported for data migration, synchronization, or change tracking with the selected feature.

3. You can create a DTS task using the visual Data Transmission Service console or an API or SDK.

   • Data synchronization operations

   • Data migration operations

   • Change Tracking Operations Guide

More information

FAQ