refactor: Update AWS Glue Databrew StartJobRun step to use integration pattern input

[ca-nguyen]

Description

Update AWS Glue Databrew service integration to use integration_pattern input instead or wait_for_completion flag.

Fixes #(issue) (N/A)

Why is the change necessary?

This change is necessary for consistency with the new service integration implementation pattern introduced in commit (Add support for Nested Step Functions) that uses the integration_pattern arg in the step constructor to build the resource.

Support for AWS Glue Databrew service integration was added in this commit, but not released yet.
A later commit (Add support for Nested Step Functions) introduced a new implementation pattern using the IntegrationPattern enum as input to construct the step instead of the wait_for_completion flag. (See PR for more detail on rationale behind the implementation).

Solution

Replace the wait_for_completion flag with integration_pattern arg in StartJobRun step construction.

The IntegrationPattern is used to build the Resource arn as follow:

IntegrationPattern	Resource	Doc
WaitForCompletion	"arn:aws:states:::states:databrew:startJobRun.sync"	Run A job
CallAndContinue	"arn:aws:states:::states:databrew:startJobRun"	Request Response

See Service Integration Patterns for more details

Normally, replacing a constructor argument would be a breaking change, but since we have not released support for AWS Glue Databrew service integration yet, it is acceptable to do so. After next release, it making such changes will be considered as not being backward compatible.

Testing

• Updated unit tests for all resources
• No integ tests added as external resources were required for testing (Databrew Job)
• Manual test done for all resources

Manual Tests

Create a Databrew job for manual tests (<databrew_job_name>).
For each test, create a workflow and execute as follow:

workflow = Workflow( name=<state_machine_name>, definition=<step>, role=<workflow_execution_role> ) workflow.create() workflow.execute() 

• Call And Continue:

# Create a workflow with the following steps & execute step_call_and_continue = GlueDataBrewStartJobRunStep( 'Start Glue DataBrew Job Run - CallAndContinue', integration_pattern=IntegrationPattern.CallAndContinue, parameters={ "Name": <databrew_job_name> } ) step_default = GlueDataBrewStartJobRunStep('Start Glue DataBrew Job Run - Default', parameters={ "Name": <databrew_job_name> } ) # Validate that the workflows end after starting the <databrew_job_name> job 

• Wait For Completion

# Create a workflow with the following step & execute step_wait_for_completion = GlueDataBrewStartJobRunStep( 'Start Glue DataBrew Job Run - WaitForCompletion', integration_pattern=IntegrationPattern.WaitForCompletion, parameters={ "Name": <databrew_job_name> } ) # Validate that the workflow only ends after the <databrew_job_name> job completes 

---

Pull Request Checklist

Please check all boxes (including N/A items)

Testing

• Unit tests added
• Integration test added - N/A
• Manual testing - why was it necessary? could it be automated? No integ tests added as external resources were required for testing

Documentation

• docs: All relevant docs updated
• docstrings: All public APIs documented

Title and description

• Change type: Title is prefixed with change type: and follows conventional commits
• References: Indicate issues fixed via: Fixes #xxx - N/A

---

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license.

Suggested changes were added in last commit

[jormello]

Nit (non-blocking): would be good to have the comment precede the Field.Resource assignment.

[StepFunctions-Bot]

AWS CodeBuild CI Report

• CodeBuild project: AutoBuildProject6AEA49D1-sEHrOdk7acJc
• Commit ID: 5c5f235
• Result: SUCCEEDED
• Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository