DataWorks:Solution 6: Connect to a data source over the Internet

If your data source meets the following condition, we recommend that you use this solution:

• A public endpoint is configured for the data source.

Step 1: Obtain basic information

Step 2: Establish a network connection

• Serverless resource group: By default, serverless resource groups cannot access the public network. To enable access to data sources on the public network, you must configure an Internet NAT gateway and an EIP for the VPC that is attached to the resource group.

• Legacy exclusive resource group: Legacy exclusive resource groups can access the Internet directly.

Step 3: (Optional) Add an IP address to a whitelist

If the data source is controlled by a whitelist, add the public IP address of the resource group to the whitelist to allow access.

This topic uses a MySQL IP address whitelist as an example. This topic describes how to specify that a user can access the database only from the public IP address that is bound to the resource group.

1. Log on to the database as an administrator.

2. Create an account to access the data source from DataWorks and grant permissions to the account.

   -- "dataworks_user" is the username. You can customize it. -- "StrongPassword123!" is the password. You can customize it. CREATE USER 'dataworks_user'@'<Public IP address bound to the resource group>' IDENTIFIED BY 'StrongPassword123!'; -- Grant the user permissions to access a specific database, such as mydatabase, from the public IP address bound to the resource group. GRANT ALL PRIVILEGES ON mydatabase.* TO 'dataworks_user'@'<Public IP address bound to the resource group>' WITH GRANT OPTION;

3. Run the FLUSH PRIVILEGES; command to refresh the permissions and then exit the database (exit).

1. Basic information

2. Establish a network connection

This solution applies only to serverless resource groups. You can use an Internet NAT gateway to enable public network access for the VPC that is bound to the resource group. Legacy resource groups are bound to EIPs by default and do not require this configuration.

1. In the DataWorks console, go to the Resource Group page. Find the target resource group and click Network Settings in the Actions column.

2. In the corresponding feature module, find the bound VPC and click next to its name to open the Basic Information page.

   For example, to connect a MySQL instance with a public endpoint to DataWorks for data synchronization, go to the Data Scheduling & Data Integration section, find the VPC, and click the icon next to its name.

   

3. Switch to the Resource Management tab. In the Public Network Access Service area, under Internet NAT Gateway, click Create Now to enable public network access for the VPC bound to the resource group.

   Configure the following key parameters:

   Parameter	Value
   VPC	Select the same VPC and vSwitch that are bound to the resource group.
   Associated VSwitch
   Access Mode	Select SNAT-enabled Mode.
   Elastic IP Address Instance	Select Purchase Elastic IP.
   Create Service-linked Role	If you are creating a NAT gateway for the first time, you must create a service-linked role. Click Create Service-linked Role.

4. Click Buy Now and complete the payment to create the NAT Gateway instance.

   

5. After the NAT Gateway instance is created, click Return to Console to create an SNAT entry for the new NAT Gateway instance.

   Note

   Resources in the VPC can access the Internet only after an SNAT entry is configured.

   1. Click Manage in the Actions column for the new instance. On the management page of the NAT Gateway instance, click the SNAT Management Tab.

   2. In the SNAT Entry List section, click Create SNAT Entry. The key parameters are described below:

      Parameter	Value
      SNAT Entry Granularity	Select VPC Granularity. This ensures that all resource groups in the VPC of the NAT gateway can access the internet through the configured EIP.
      Select Elastic IP Address	Select the EIP that is bound to the current NAT Gateway instance.

      After you configure the parameters for the SNAT entry, click OK to create the entry.

   In the SNAT Entry List, when the Status of the new SNAT entry changes to Available, the VPC bound to the resource group can access the internet.

3. Add an IP address to a whitelist

1. Obtain the public IP address that is bound to the resource group.

   Serverless resource group

   1. Go to the VPC console. In the navigation pane on the left, choose NAT Gateway > Internet NAT Gateway.

   2. Find the Internet NAT gateway that you created. In the Elastic IP Address column, you can find the Elastic IP Address.

      

   Legacy resource group

   1. Go to the Resource Group page in the DataWorks console. Find the target resource group and click Details in the Actions column to go to the details page of the resource group.

   2. Obtain the EIP address.

      

2. Log on to the database as an administrator.

3. Create an account to access the data source from DataWorks and grant permissions to the account.

   -- "dataworks_user" is the username. You can customize it. -- "StrongPassword123!" is the password. You can customize it. CREATE USER 'dataworks_user'@'<Public IP address bound to the resource group>' IDENTIFIED BY 'StrongPassword123!'; -- Grant the user permissions to access a specific database, such as mydatabase, from the public IP address bound to the resource group. GRANT ALL PRIVILEGES ON mydatabase.* TO 'dataworks_user'@'<Public IP address bound to the resource group>' WITH GRANT OPTION;

4. Run the FLUSH PRIVILEGES; command to refresh the permissions and then exit the database (exit).

4. Test the network connectivity

1. Log on to the DataWorks console. In the top navigation bar, select the desired region. In the left-side navigation pane, choose Data Integration > Data Integration. On the page that appears, select the desired workspace from the drop-down list and click Go to Data Integration.

2. In the navigation pane on the left, click Data Source. On the Data Source List page, click Add Data Source.

3. Select the MySQL data source type and configure its parameters.

   • Set Configuration Mode to Connection String Mode.

   • Set Host Address ID to the public IP address of the MySQL server. In this example, the IP address is 47.117.XX.XX.

   • Set Port Number to 3306.

   • For Database Name, specify the name of an existing database.

   • Username and Password: Enter the username and password for the dataworks_user account that you created in the 3. Configure the IP address whitelist of the data source step.

4. In the Connection Configuration section, click Test Connectivity for the resource group bound to the workspace. Verify that the result is Connected.

   

   Note

   If the test fails, you can click Self-service Troubleshoot to resolve the issue. If the test still fails after troubleshooting, submit a ticket.