✨ Allow configuring "options=" in source-postgres jdbc url #29534

postamar · 2023-08-17T17:22:21Z

What

The postgres source correctly handles the options JDBC URL parameter.

How

Specialize the generic JDBC URL parameter parse logic for the postgres use case, similarly to #25859

Closes #29238

github-actions · 2023-08-17T17:22:46Z

Before Merging a Connector Pull Request

Wow! What a great pull request you have here! 🎉

To merge this PR, ensure the following has been done/considered for each connector added or updated:

PR name follows PR naming conventions
Breaking changes are considered. If a Breaking Change is being introduced, ensure an Airbyte engineer has created a Breaking Change Plan.
Connector version has been incremented in the Dockerfile and metadata.yaml according to our Semantic Versioning for Connectors guidelines
You've updated the connector's metadata.yaml file any other relevant changes, including a breakingChanges entry for major version bumps. See metadata.yaml docs
Secrets in the connector's spec are annotated with airbyte_secret
All documentation files are up to date. (README.md, bootstrap.md, docs.md, etc...)
Changelog updated in docs/integrations/<source or destination>/<name>.md with an entry for the new version. See changelog example
Migration guide updated in docs/integrations/<source or destination>/<name>-migrations.md with an entry for the new version, if the version is a breaking change. See migration guide example
If set, you've ensured the icon is present in the platform-internal repo. (Docs)

If the checklist is complete, but the CI check is failing,

Check for hidden checklists in your PR description
Toggle the github label checklist-action-run on/off to re-run the checklist CI.

github-actions · 2023-08-17T17:47:15Z

Coverage report for source-postgres

File	Coverage [88.67%]	🍏
PostgresSource.java	88.67%	🍏

Total Project Coverage	70.19%	🍏

github-actions · 2023-08-17T20:39:54Z

source-alloydb-strict-encrypt test report (commit `19a3013d30`) - ❌

⏲️ Total pipeline duration: 06mn28s

Step	Result
Validate airbyte-integrations/connectors/source-alloydb-strict-encrypt/metadata.yaml	✅
Connector version semver check	✅
Connector version increment check	✅
QA checks	✅
Build connector tar	✅
Build source-alloydb-strict-encrypt docker image for platform linux/x86_64	✅
./gradlew :airbyte-integrations:connectors:source-alloydb-strict-encrypt:integrationTest	✅
Acceptance tests	❌

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-alloydb-strict-encrypt test

github-actions · 2023-08-17T20:44:54Z

source-alloydb test report (commit `19a3013d30`) - ❌

⏲️ Total pipeline duration: 04mn50s

Step	Result
Validate airbyte-integrations/connectors/source-alloydb/metadata.yaml	✅
Connector version semver check	✅
Connector version increment check	✅
QA checks	✅
Build connector tar	✅
Build source-alloydb docker image for platform linux/x86_64	✅
./gradlew :airbyte-integrations:connectors:source-alloydb:integrationTest	✅
Acceptance tests	❌

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-alloydb test

github-actions · 2023-08-17T20:51:39Z

source-postgres-strict-encrypt test report (commit `19a3013d30`) - ❌

⏲️ Total pipeline duration: 06mn34s

Step	Result
Validate airbyte-integrations/connectors/source-postgres-strict-encrypt/metadata.yaml	✅
Connector version semver check	✅
Connector version increment check	✅
QA checks	✅
Build connector tar	✅
Build source-postgres-strict-encrypt docker image for platform linux/x86_64	✅
./gradlew :airbyte-integrations:connectors:source-postgres-strict-encrypt:integrationTest	✅
Acceptance tests	❌

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-postgres-strict-encrypt test

github-actions · 2023-08-17T21:04:34Z

source-postgres test report (commit `19a3013d30`) - ✅

⏲️ Total pipeline duration: 12mn44s

Step	Result
Validate airbyte-integrations/connectors/source-postgres/metadata.yaml	✅
Connector version semver check	✅
Connector version increment check	✅
QA checks	✅
Build connector tar	✅
Build source-postgres docker image for platform linux/x86_64	✅
./gradlew :airbyte-integrations:connectors:source-postgres:integrationTest	✅
Acceptance tests	✅

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-postgres test

github-actions · 2023-08-21T18:26:05Z

source-alloydb-strict-encrypt test report (commit `19a3013d30`) - ❌

⏲️ Total pipeline duration: 06mn10s

Step	Result
Validate airbyte-integrations/connectors/source-alloydb-strict-encrypt/metadata.yaml	✅
Connector version semver check	✅
Connector version increment check	✅
QA checks	✅
Build connector tar	✅
Build source-alloydb-strict-encrypt docker image for platform linux/x86_64	✅
./gradlew :airbyte-integrations:connectors:source-alloydb-strict-encrypt:integrationTest	✅
Acceptance tests	❌

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-alloydb-strict-encrypt test

github-actions · 2023-08-21T18:39:06Z

source-postgres test report (commit `19a3013d30`) - ✅

⏲️ Total pipeline duration: 12mn51s

Step	Result
Validate airbyte-integrations/connectors/source-postgres/metadata.yaml	✅
Connector version semver check	✅
Connector version increment check	✅
QA checks	✅
Build connector tar	✅
Build source-postgres docker image for platform linux/x86_64	✅
./gradlew :airbyte-integrations:connectors:source-postgres:integrationTest	✅
Acceptance tests	✅

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-postgres test

github-actions · 2023-08-21T18:45:26Z

source-postgres-strict-encrypt test report (commit `19a3013d30`) - ✅

⏲️ Total pipeline duration: 06mn09s

Step	Result
Validate airbyte-integrations/connectors/source-postgres-strict-encrypt/metadata.yaml	✅
Connector version semver check	✅
Connector version increment check	✅
QA checks	✅
Build connector tar	✅
Build source-postgres-strict-encrypt docker image for platform linux/x86_64	✅
./gradlew :airbyte-integrations:connectors:source-postgres-strict-encrypt:integrationTest	✅
Acceptance tests	✅

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-postgres-strict-encrypt test

github-actions · 2023-08-21T18:50:10Z

source-alloydb test report (commit `19a3013d30`) - ❌

⏲️ Total pipeline duration: 04mn33s

Step	Result
Validate airbyte-integrations/connectors/source-alloydb/metadata.yaml	✅
Connector version semver check	✅
Connector version increment check	✅
QA checks	✅
Build connector tar	✅
Build source-alloydb docker image for platform linux/x86_64	✅
./gradlew :airbyte-integrations:connectors:source-alloydb:integrationTest	✅
Acceptance tests	❌

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-alloydb test

postamar · 2023-08-22T14:16:54Z

Thanks for the review. I'm still deep-diving into how acceptance tests work. I'll want to make the source-postgres-strict-encrypt acceptance tests meaningful, right now the config yaml is little more than a stub.

github-actions · 2023-08-22T22:34:12Z

source-alloydb test report (commit `2730386ecf`) - ❌

⏲️ Total pipeline duration: 07mn46s

Step	Result
Java Connector Unit Tests	✅
Build connector tar	✅
Build source-alloydb docker image for platform linux/x86_64	✅
Java Connector Integration Tests	✅
Acceptance tests	❌
Validate airbyte-integrations/connectors/source-alloydb/metadata.yaml	✅
Connector version semver check	✅
Connector version increment check	✅
QA checks	✅

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-alloydb test

github-actions · 2023-08-22T22:40:44Z

source-alloydb-strict-encrypt test report (commit `2730386ecf`) - ❌

⏲️ Total pipeline duration: 06mn22s

Step	Result
Java Connector Unit Tests	✅
Build connector tar	✅
Build source-alloydb-strict-encrypt docker image for platform linux/x86_64	✅
Java Connector Integration Tests	✅
Acceptance tests	❌
Validate airbyte-integrations/connectors/source-alloydb-strict-encrypt/metadata.yaml	✅
Connector version semver check	✅
Connector version increment check	✅
QA checks	✅

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-alloydb-strict-encrypt test

github-actions · 2023-08-22T22:53:07Z

source-postgres-strict-encrypt test report (commit `2730386ecf`) - ✅

⏲️ Total pipeline duration: 12mn12s

Step	Result
Java Connector Unit Tests	✅
Build connector tar	✅
Build source-postgres-strict-encrypt docker image for platform linux/x86_64	✅
Java Connector Integration Tests	✅
Acceptance tests	✅
Validate airbyte-integrations/connectors/source-postgres-strict-encrypt/metadata.yaml	✅
Connector version semver check	✅
Connector version increment check	✅
QA checks	✅

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-postgres-strict-encrypt test

github-actions · 2023-08-22T23:26:23Z

source-postgres test report (commit `2730386ecf`) - ✅

⏲️ Total pipeline duration: 33mn04s

Step	Result
Java Connector Unit Tests	✅
Build connector tar	✅
Build source-postgres docker image for platform linux/x86_64	✅
Java Connector Integration Tests	✅
Acceptance tests	✅
Validate airbyte-integrations/connectors/source-postgres/metadata.yaml	✅
Connector version semver check	✅
Connector version increment check	✅
QA checks	✅

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-postgres test

postamar · 2023-08-23T18:11:46Z

/approve-and-merge reason="source-alloydb is a thin wrapper around source-postgres"

octavia-approvington · 2023-08-23T18:12:18Z

This is really good

…#29534)

Marius Posta added 3 commits August 17, 2023 10:19

add support for 'options' JDBC URL param

5aac828

bump versions for PostgresSource sources

bca268e

update changelogs in docs

bd363fb

octavia-squidington-iii added area/connectors Connector related issues area/documentation Improvements or additions to documentation connectors/source/postgres connectors/source/postgres-strict-encrypt connectors/source/alloydb connectors/source/alloydb-strict-encrypt labels Aug 17, 2023

update doc

19a3013

postamar marked this pull request as ready for review August 17, 2023 20:31

postamar requested a review from a team as a code owner August 17, 2023 20:31

rodireich approved these changes Aug 22, 2023

View reviewed changes

Marius Posta added 3 commits August 22, 2023 13:47

add meaningful acceptance test configs

58d0fb1

Merge branch 'master' into fix-29238-attempt-3

54d0f97

fix spec and bump versions

2730386

octavia-approvington approved these changes Aug 23, 2023

View reviewed changes

octavia-approvington merged commit a26a4ad into master Aug 23, 2023

octavia-approvington deleted the fix-29238-attempt-3 branch August 23, 2023 18:12

harrytou pushed a commit to KYVENetwork/airbyte that referenced this pull request Sep 1, 2023

✨ Allow configuring "options=" in source-postgres jdbc url (airbytehq…

63860c8

…#29534)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

✨ Allow configuring "options=" in source-postgres jdbc url #29534

✨ Allow configuring "options=" in source-postgres jdbc url #29534

Uh oh!

postamar commented Aug 17, 2023 •

edited

Loading

github-actions bot commented Aug 17, 2023 •

edited by postamar

Loading

github-actions bot commented Aug 17, 2023 •

edited

Loading

github-actions bot commented Aug 17, 2023

github-actions bot commented Aug 17, 2023

github-actions bot commented Aug 17, 2023

github-actions bot commented Aug 17, 2023

github-actions bot commented Aug 21, 2023

github-actions bot commented Aug 21, 2023

github-actions bot commented Aug 21, 2023

github-actions bot commented Aug 21, 2023

postamar commented Aug 22, 2023

github-actions bot commented Aug 22, 2023

github-actions bot commented Aug 22, 2023

github-actions bot commented Aug 22, 2023

github-actions bot commented Aug 22, 2023

postamar commented Aug 23, 2023

octavia-approvington commented Aug 23, 2023

Labels

5 participants

✨ Allow configuring "options=" in source-postgres jdbc url #29534

✨ Allow configuring "options=" in source-postgres jdbc url #29534

Uh oh!

Conversation

postamar commented Aug 17, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

How

github-actions bot commented Aug 17, 2023 • edited by postamar Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Before Merging a Connector Pull Request

github-actions bot commented Aug 17, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Coverage report for source-postgres

github-actions bot commented Aug 17, 2023

source-alloydb-strict-encrypt test report (commit 19a3013d30) - ❌

github-actions bot commented Aug 17, 2023

source-alloydb test report (commit 19a3013d30) - ❌

github-actions bot commented Aug 17, 2023

source-postgres-strict-encrypt test report (commit 19a3013d30) - ❌

github-actions bot commented Aug 17, 2023

source-postgres test report (commit 19a3013d30) - ✅

github-actions bot commented Aug 21, 2023

source-alloydb-strict-encrypt test report (commit 19a3013d30) - ❌

github-actions bot commented Aug 21, 2023

source-postgres test report (commit 19a3013d30) - ✅

github-actions bot commented Aug 21, 2023

source-postgres-strict-encrypt test report (commit 19a3013d30) - ✅

github-actions bot commented Aug 21, 2023

source-alloydb test report (commit 19a3013d30) - ❌

postamar commented Aug 22, 2023

github-actions bot commented Aug 22, 2023

source-alloydb test report (commit 2730386ecf) - ❌

github-actions bot commented Aug 22, 2023

source-alloydb-strict-encrypt test report (commit 2730386ecf) - ❌

github-actions bot commented Aug 22, 2023

source-postgres-strict-encrypt test report (commit 2730386ecf) - ✅

github-actions bot commented Aug 22, 2023

source-postgres test report (commit 2730386ecf) - ✅

postamar commented Aug 23, 2023

octavia-approvington commented Aug 23, 2023

Labels

5 participants

postamar commented Aug 17, 2023 •

edited

Loading

github-actions bot commented Aug 17, 2023 •

edited by postamar

Loading

github-actions bot commented Aug 17, 2023 •

edited

Loading

source-alloydb-strict-encrypt test report (commit `19a3013d30`) - ❌

source-alloydb test report (commit `19a3013d30`) - ❌

source-postgres-strict-encrypt test report (commit `19a3013d30`) - ❌

source-postgres test report (commit `19a3013d30`) - ✅

source-alloydb-strict-encrypt test report (commit `19a3013d30`) - ❌

source-postgres test report (commit `19a3013d30`) - ✅

source-postgres-strict-encrypt test report (commit `19a3013d30`) - ✅

source-alloydb test report (commit `19a3013d30`) - ❌

source-alloydb test report (commit `2730386ecf`) - ❌

source-alloydb-strict-encrypt test report (commit `2730386ecf`) - ❌

source-postgres-strict-encrypt test report (commit `2730386ecf`) - ✅

source-postgres test report (commit `2730386ecf`) - ✅