Destination:PubSub Add Batching and Ordering configuration

[Kopiczek]

What

Previously PubSub publisher was always using ordering feature and had batching disabled.
This change allows to configure this on the destination configuration level.
Batching is especially useful for large syncs.

How

Describe the solution

Recommended reading order

Any

🚨 User Impact 🚨

No, configuration change should be backward compatible AFAIK (since we have default values).

Pre-merge Checklist

Expand the relevant checklist and delete the others.

Updating a connector

Community member or Airbyter

• Grant edit access to maintainers (instructions)
• Secrets in the connector's spec are annotated with airbyte_secret
• Unit & integration tests added and passing. Community members, please provide proof of success locally e.g: screenshot or copy-paste unit, integration, and acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
• Code reviews completed
• Documentation updated
  • Connector's README.md
  • Connector's bootstrap.md. See description and examples
  • Changelog updated in docs/integrations/<source or destination>/<name>.md including changelog. See changelog example
• PR name follows PR naming conventions

Tests

Unit

Put your unit tests output here.

Integration

Execution optimizations have been disabled for 4 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD SUCCESSFUL in 13m 57s
74 actionable tasks: 7 executed, 67 up-to-date
4:08:19 p.m.: Execution finished 'integrationTest'.

Acceptance

Put your acceptance tests output here.

[Kopiczek]

@marcosmarxm Hi Marcos, since you looked at my last PR, would you be ok to see this one as well?

[marcosmarxm]

/test connector=connectors/destination-pubsub

🕑 connectors/destination-pubsub https://github.com/airbytehq/airbyte/actions/runs/2952036794
❌ connectors/destination-pubsub https://github.com/airbytehq/airbyte/actions/runs/2952036794
🐛 https://gradle.com/s/73uh747mno2rw

Build Failed

Test summary info:

Could not find result summary 

[marcosmarxm]

Sorry the delay here @Kopiczek team will take a look this week or the next one.

[Kopiczek]

@marcosmarxm any news? been almost 3 weeks now :)
Not sure why the tests in github failed, locally it works.
This might be some out-of-order issue?

[marcosmarxm]

@Kopiczek sorry the delay, I'll add to my priority list review and return to you!

[Kopiczek]

hey @marcosmarxm , any updates?

[marcosmarxm]

@Kopiczek sorry the delay to merge this, tomorrow I'll review and solve issues for this contribution!

[Kopiczek]

@marcosmarxm ping

[marcosmarxm]

@Kopiczek sorry, we received a lot of contributions for hacktoberfest and prioritize them. Next week we're going to return to other contirbutions. Sorry again!

[marcosmarxm]

Hello 👋, first thank you for this amazing contribution.

We really appreciate the effort you've made to improve the project.
We ask you patience for the code review. Last month our team was focused on Hacktoberfest event and that probably left some PR without the proper feedback. And this week, due to the Thanksgiving US Holiday, most our team is out of office with their families. Another important piece of information why code won't be merge this week is: as a safety measure the core team has decided to freeze merging code to main branch to keep the release stable. Next week we'll return to you with the proper code review and update the status of your contribution.

If you have any questions feel free to send me a message in Slack!
Thanks!

[marcosmarxm]

/test connector=connectors/destination-pubsub

🕑 connectors/destination-pubsub https://github.com/airbytehq/airbyte/actions/runs/3605384055
❌ connectors/destination-pubsub https://github.com/airbytehq/airbyte/actions/runs/3605384055
🐛

Build Passed

Test summary info:

All Passed 

[marcosmarxm]

/test connector=connectors/destination-pubsub

🕑 connectors/destination-pubsub https://github.com/airbytehq/airbyte/actions/runs/3605503012
✅ connectors/destination-pubsub https://github.com/airbytehq/airbyte/actions/runs/3605503012
No Python unittests run

Build Passed

Test summary info:

All Passed 

[marcosmarxm]

/publish connector=connectors/destination-pubsub

🕑 Publishing the following connectors:
connectors/destination-pubsub
https://github.com/airbytehq/airbyte/actions/runs/3605618187

Connector	Did it publish?	Were definitions generated?
connectors/destination-pubsub	✅	✅

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

[marcosmarxm]

Thanks @Kopiczek so sorry the delay to publish it