This topic uses an ApsaraDB RDS for MySQL instance as an example to describe how to establish a network connection between a DataWorks workspace and a data source that are in different regions under the same Alibaba Cloud account.
Scenarios
If your data source and resource group meet the following conditions at the same time, we recommend that you use this solution:
The data source belongs to an Alibaba Cloud service.
The data source and resource group belong to the same Alibaba Cloud account.
The data source and resource group reside in different regions.
Solution description
If your data source and workspace are in different regions under the same account, you can connect them using a private VPC network. You can use a network connectivity tool, such as Cloud Enterprise Network (CEN) or a VPC peering connection, to connect the VPC of the DataWorks resource group to the VPC of the data source.
Prerequisites
The data source that you want to use is added to DataWorks. For information about data source types that are supported by DataWorks, see Supported data sources and sync solutions.
A workspace is created. For more information, see Create a workspace.
A serverless resource group is created and associated with your workspace. For more information, see 1. Create a serverless resource group and 2. Associate the resource group with a workspace.
The data source and DataWorks workspace meet the requirements in the Scenarios section.
Billing
The billing method varies based on the network connectivity tool that you use. For more information, see Billing of CEN or Billing of VPC peering connections.
Configure network connectivity
This section describes the general procedure for configuring network connectivity between a data source and DataWorks. This helps you understand the core logic. For detailed configuration steps, see the Configuration example section in this topic.
Step 1: Obtain basic information
Data source side
Region information: In this example, an ApsaraDB RDS for MySQL instance in the China (Hangzhou) region is used.
VPC and vSwitch information:
NoteThis topic uses an ApsaraDB RDS for MySQL instance as an example. For other Alibaba Cloud instances, see the official documentation to learn how to obtain VPC information.
Go to the RDS console, locate the target instance, and click the Instance Name to open the Basic Information page.
In the navigation pane on the left, click Database Connection. On the page that appears, find the VPC and vSwitch information for the ApsaraDB RDS for MySQL instance.
DataWorks side
Region information: In this example, a DataWorks workspace and a resource group in the China (Shanghai) region are used.
VPC and vSwitch information of the attached resource group:
On the DataWorks Resource Groups page, find the target resource group and click Network Settings in the Actions column.
In the corresponding feature module, you can view the attached VPC and VSwitch information.
For example, if you want to connect an RDS for MySQL instance to DataWorks for data synchronization, you can find the required VPC and VSwitch information in the Data Scheduling & Data Integration section.
Step 2: Establish a network connection
To connect VPCs across regions, you can use a network connectivity tool. The following options are available:
Cloud Enterprise Network: Suitable for interconnecting multiple VPCs in complex enterprise network environments. For more information, see Connect VPCs across regions.
VPC peering connection: Suitable for interconnecting two VPCs. For more information, see Use a VPC peering connection to achieve private connectivity between VPCs.
If you encounter issues when you configure the network connection, you can submit a ticket to contact technical support for the relevant cloud product.
Step 3: Add a route for the DataWorks resource group
When DataWorks accesses a data source in a different region, you must also add a route in the DataWorks resource group. The route must point to the vSwitch CIDR block where the data source is located.
On the Resource Groups page of the DataWorks console, find the target resource group, and click Network Settings in the Actions column.
In the corresponding feature module, find the attached VPC, and in its Actions column, click Custom Route.
Click Add Route. Set the connection method to CIDR Block and Destination CIDR Block to the vSwitch CIDR block of the data source.
Step 4: (Optional) Add to whitelist
If the data source uses a whitelist for access control, you must add the vSwitch CIDR block of the attached resource group to the data source's whitelist to allow access.
This topic provides an example of setting an IP whitelist for an RDS for MySQL instance. In the Whitelist and Security Group tab, add the VSwitch CIDR Block that is attached to the DataWorks resource group.
For other Alibaba Cloud instances, see the official documentation to learn how to configure a whitelist.
Test network connectivity
Log on to the DataWorks console. In the top navigation bar, select the desired region. In the left-side navigation pane, choose . On the page that appears, select the desired workspace from the drop-down list and click Go to Data Integration.
In the left navigation pane, click Data Source. On the Data Sources page, click Add Data Source. Select a data source and configure its connection parameters.
In the resource group list at the bottom of the page, select the resource group that is connected to the data source and click Test Connectivity.
NoteIf the connectivity test Fails, you can use the Connectivity Diagnosis Tool to identify and resolve the connection problem. If the problem persists, submit a ticket.
Configuration example
This section provides an example of how to configure a network connection between an ApsaraDB RDS for MySQL instance in the China (Hangzhou) region and a DataWorks workspace in the China (Shanghai) region. Both the instance and the workspace belong to the same Alibaba Cloud account.
1. Basic information
Parameter | Data source (RDS for MySQL) | DataWorks resource group |
Account | Same account | |
Region | China (Hangzhou) | China (Shanghai) |
VPC |
|
|
2. Establish a network connection
You can use Cloud Enterprise Network (CEN) or a VPC peering connection to connect the data source to DataWorks. Choose a method based on your requirements.
If you encounter issues when you configure the network connection, you can submit a ticket to contact technical support for the relevant cloud product.
Configure a network connection using CEN
Log on to the Cloud Enterprise Network console. On the Instances page, click Create CEN Instance.
In the Create CEN Instance dialog box, on the Scenario-based Creation (Recommended) tab, select the Cloud-based Multi-VPC Interconnection scenario and click Start Scenario-based Creation.
NoteThe Cloud-based Multi-VPC Interconnection scenario interconnects VPCs within a single account, in the same region or across different regions.
In the Configure Networking Settings step, configure the network settings for the data source and the DataWorks resource group.
On the first tab, configure the network information for the data source:
For Region, select China (Hangzhou).
VPC:
VPC: Select the VPC where the data source is deployed. This example uses
vpc_hangzhou
.VSwitch: Select the vSwitch where the data source is deployed. For example,
hz-h
is selected.NoteNetwork interconnection using Cloud Enterprise Network (CEN) requires zone-level disaster recovery. Therefore, you must configure at least two vSwitches in different zones. One vSwitch must be the one where the data source is deployed, and the other must be in a different zone. If a suitable vSwitch is not available, you can click Create VSwitch from the drop-down menu to create a new one.
Click the + icon to add a new tab and configure the network information for the DataWorks resource group:
For Region, select China (Shanghai).
VPC:
VPC: Select the VPC where the DataWorks resource group is deployed. In this example,
vpc_shanghai
is selected.VSwitch: Select the vSwitch where the DataWorks resource group is deployed. This example uses
sh-l
.NoteFor zone-level disaster recovery, network interconnection using Cloud Enterprise Network (CEN) requires you to configure at least two vSwitches in different zones: one in the resource group's zone and another in a different zone. If a suitable vSwitch is not available, click Create VSwitch in the drop-down menu to create one, and then select the created vSwitch.
Click Next. CEN automatically generates the Network Configuration and Billing Details based on the parameters from the previous step. Confirm that the information is correct, and then click Start Deployment.
Wait for the deployment to be completed.
ImportantDo not close the page until the deployment is complete.
Configure a network connection using a VPC peering connection
Go to the VPC Peering Connection console. At the top of the page, switch the region to China (Hangzhou). Then, click Create Peering Connection and configure the parameters.
The following table describes the key parameters for this example. You can keep the default values for other parameters.
Parameter
Description and example
Peering Connection Name
Specify a name as needed. In this example, the following name is specified:
Interconnection between VPCs in the China (Hangzhou) and China (Shanghai) regions within the same Alibaba Cloud account
.Requester VPC
Select the VPC in which the ApsaraDB RDS for MySQL instance is deployed. In this example,
vpc-hangzhou
is selected.Accepter Account Type
In this example,
Current Account
is selected.Accepter Region Type
In this example,
Inter-Region
is selected.Accepter Region
The region where the DataWorks workspace and resource group are located. Select
China (Shanghai)
.Accepter VPC
Select the VPC in which the DataWorks resource group is deployed. In this example,
vpc-shanghai
is selected.Click OK to finish configuring the peering connection.
In the VPC Peering Connection console, find the created peering connection, and click Configure Route in the Requester VPC and Accepter VPC columns.
Configure a route entry for the requester VPC to point to the accepter VPC: Specify a custom Name and set the Destination CIDR Block to the vSwitch CIDR block of the accepter VPC. In this example, the destination CIDR block is
172.16.0.0/24
.Configure a route entry for the accepter VPC, pointing to the requester VPC: Specify a custom Name and set the Destination CIDR Block to the vSwitch CIDR block of the requester VPC, such as
192.168.6.0/24
.
3. Add a route for the DataWorks resource group
Go to the Resource Groups page of the DataWorks console. Find the target resource group and click Network Settings in the Actions column.
Find the VPC attached to the feature and click Custom Route in the Actions column.
Click Add Route, select Specify CIDR Block as the Connection Method, and set Destination CIDR Block to the vSwitch CIDR block of the RDS for MySQL instance (in this example,
192.168.6.0/24
).
4. Configure whitelist
Add the VSwitch CIDR Block associated with the DataWorks resource group to the Whitelist and Security Group settings of your RDS for MySQL instance. In this example, the CIDR block is 172.16.0.0/24
.
5. Test network connectivity
Log on to the DataWorks console. In the top navigation bar, select the desired region. In the left-side navigation pane, choose . On the page that appears, select the desired workspace from the drop-down list and click Go to Data Integration.
In the navigation pane on the left, click Data Source. On the Data Sources page, click Add Data Source.
Select MySQL as the data source type and configure its parameters.
Set the Configuration Mode parameter to Alibaba Cloud Instance Mode.
Set the Region parameter to China (Hangzhou).
Instance: You can select the ApsaraDB RDS for MySQL instance from the China (Hangzhou) region that has passed the network connectivity test.
In the Connection Configuration section, click Test Connectivity for the resource group bound to the workspace, and verify that the status is Connected.
NoteIf the connectivity test Fails, you can use the Connectivity Diagnosis Tool to identify and resolve the connection problem. If the problem persists, submit a ticket.
References
For information about common issues related to network connectivity, see Resource group operations and network connectivity.