DataWorks:Extensions

Limits

• Only users of DataWorks Enterprise Edition can use the Extensions module.

• The Extensions module is available in the following regions: China (Beijing), China (Hangzhou), China (Shanghai), China (Zhangjiakou), China (Shenzhen), China (Chengdu), US (Silicon Valley), US (Virginia), Germany (Frankfurt), Japan (Tokyo), China (Hong Kong), and Singapore.

Notes

• Permission control: Only the Open Platform administrator, tenant administrator, Alibaba Cloud accounts, and RAM users to which the AliyunDataWorksFullAccess policy is attached have read and write permissions on the developer backend. For more information about permission management, see Manage permissions on global-level services and Manage permissions on the DataWorks services and the entities in the DataWorks console by using RAM policies.

• Version validity period: If DataWorks Enterprise Edition expires, extensions become invalid and cannot be triggered to check extension point events. If an extension is triggered to check an event and has not completed the check when DataWorks Enterprise Edition expires, the check is terminated and the result Check Passed is returned.

• Limits: If you develop and deploy an extension based on Function Compute, you can add only specific extension point events to the extension.

Features

DataWorks Open Platform provides various extension points. The Extensions module is based on extension points. The Extensions module is a plug-in that works with the OpenAPI and OpenEvent modules to provide features that can meet your business requirements and help you manage the operations you perform in DataWorks based on custom processing logic, such as blocking the operations.

Business scenarios in which extensions can be used for operation management:

• Management of table or task naming conventions.

• Management of duplicate data synchronization tasks.

• Management of fees generated by tasks.

• Management of dependencies between tasks.

  Note

  For more information about the event types that can be used in various business scenarios, see Development reference: Event lists and event message formats.

Custom process control: When an extension is used to control an operation at an extension point, the operation in DataWorks is suspended until the custom extension returns a result.

Development process

In DataWorks, you can develop and deploy extensions through Self-managed Service or Function Compute to implement custom process control.

• Configure the required settings in OpenEvent to push event messages: Select a method to develop and deploy an extension based on the extension point events that you want to receive.

  • Develop and deploy an extension based on Self-managed Service: This method relies on the message distribution capabilities of EventBridge. Specify the event bus to which DataWorks event messages are sent in the configured event distribution channel. Then, configure the event bus to route the messages to the appropriate type of service.

  • Develop and deploy an extension by using Function Compute: By default, DataWorks event messages are sent to the Function Compute service that you specify in the extension that you register. You do not need to configure an event bus in OpenEvent.

• Develop and deploy an extension: Receive and parse event messages pushed by DataWorks, process the event messages based on custom processing logic, and return the processing result to DataWorks.

  • Develop and deploy an extension based on a self-managed service: Use the service program to which messages are ultimately routed in the event bus to receive DataWorks messages, make logical judgments on the messages, and call back the processing results by using the specified API.

  • Develop and deploy an extension based on Function Compute: Implement a specific interface of Function Compute to receive extension requests and return extension processing results.

• Register the extension: Define which types of event messages the developed extension will receive and process in DataWorks.

• Test the extension: Verify whether the extension works as expected in the test workspace.

• Submit and publish the extension: After you confirm that the extension works as expected, submit the extension for review. After the extension passes the review, publish the extension for all workspaces to use.

  Note

  • The review of an extension is usually completed within T+3 working days. Please be patient.

  • DataWorks event messages are valid for three days. If the validity period is exceeded, the event messages are considered expired and are not processed by an extension.

Supported extension point events

The following table describes the types and details of extension point events that can be processed by extensions.

Appendix: Comparison between two types of extension deployment methods