- Notifications
You must be signed in to change notification settings - Fork 4.9k
Source Stripe: add availability strategy #22659
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Source Stripe: add availability strategy #22659
Conversation
erohmensing left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Solid - this sort of work would have handled the bug I ran into the first time around 😄 Please add a unit test!
airbyte-integrations/connectors/source-stripe/source_stripe/availability_strategy.py Show resolved Hide resolved
| available, reason = availability_strategy.check_availability(parent_stream_instance, logger, source) | ||
| if not available: | ||
| return available, reason |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| available, reason = availability_strategy.check_availability(parent_stream_instance, logger, source) | |
| if not available: | |
| return available, reason | |
| is_available, reason = availability_strategy.check_availability(parent_stream_instance, logger, source) | |
| if not is_available: | |
| return is_available, reason |
Small suggestion to make return is_available, reason not look on first glance that it's returning a true
airbyte-integrations/connectors/source-stripe/source_stripe/availability_strategy.py Show resolved Hide resolved
erohmensing left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oops I meant to request changes for the test
erohmensing left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the unit tests! Last question is about empty streams
airbyte-integrations/connectors/source-stripe/source_stripe/availability_strategy.py Show resolved Hide resolved
| empty_streams: ["bank_accounts", "checkout_sessions", "checkout_sessions_line_items", "external_account_bank_accounts"] | ||
| # TEST 1 - Reading catalog without invoice_line_items | ||
| - config_path: "secrets/config.json" | ||
| configured_catalog_path: "integration_tests/non_invoice_line_items_catalog.json" | ||
| empty_streams: ["transfers"] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What led to these increases in empty streams?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We have new account for stripe. It was populated recently by our unblockers team. There are streams which they cannot populate right now for various reasons:
bank_accounts external_account_bank_accounts checkout_sessions checkout_sessions_line_items transfers In order to not to block development of this connector we decided to create a new task to populate those streams later. I can do nothing with it for now
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok, thanks for explaining, as long as that's being tracked!
airbyte-integrations/connectors/source-stripe/source_stripe/availability_strategy.py Outdated Show resolved Hide resolved
airbyte-integrations/connectors/source-stripe/unit_tests/test_availability_strategy.py Outdated Show resolved Hide resolved
| /test connector=connectors/source-stripe
Build PassedTest summary info: |
| /publish connector=connectors/source-stripe
if you have connectors that successfully published but failed definition generation, follow step 4 here |
Airbyte Code Coverage
|
cd85519 to b93c1fe Compare fix changelog message Co-authored-by: Ella Rohm-Ensing <erohmensing@gmail.com>
…availability_strategy.py assert availability result Co-authored-by: Ella Rohm-Ensing <erohmensing@gmail.com>
| /approve-and-merge reason="published before code freeze" |

What
- turn availability strategy for all streams
- prepare custom availability strategy for substreams