🐛 Source Zendesk Support: refactor TicketComments stream #11237

bazarnov · 2022-03-17T16:17:11Z

What

Resolving: #8519

How

edited stream.py by refactoring the TicketComments stream to fetch the data from incremental/ticket_events endpoint
edited TicketComments stream schema to include missing properties, after refactoring

🚨 User Impact 🚨

No impact expected.

Pre-merge Checklist

Expand the relevant checklist and delete the others.

Updating a connector

Community member or Airbyter

Unit & integration tests added and passing. Community members, please provide proof of success locally e.g: screenshot or copy-paste unit, integration, and acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
Code reviews completed
Documentation updated
- Changelog updated in docs/integrations/<source or destination>/<name>.md including changelog. See changelog example
PR name follows PR naming conventions

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

Create a non-forked branch based on this PR and test the below items on it
Build is successful
If new credentials are required for use in CI, add them to GSM. Instructions.
/test connector=connectors/<name> command is passing
New Connector version released on Dockerhub by running the /publish command described here
After the new connector version is published, connector version bumped in the seed directory as described here
Seed specs have been re-generated by building the platform and committing the changes to the seed spec files, as described here

codecov · 2022-03-17T16:42:08Z

Codecov Report

❗ No coverage uploaded for pull request base (master@3d9f9ec). Click here to learn what that means.
The diff coverage is n/a.

@@ Coverage Diff @@ ## master #11237 +/- ## ========================================= Coverage ? 75.22% ========================================= Files ? 3 Lines ? 331 Branches ? 0 ========================================= Hits ? 249 Misses ? 82 Partials ? 0

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3d9f9ec...a2bc55c. Read the comment docs.

bazarnov · 2022-03-17T17:13:59Z

/test connector=connectors/source-zendesk-support

🕑 connectors/source-zendesk-support https://github.com/airbytehq/airbyte/actions/runs/1999805755
✅ connectors/source-zendesk-support https://github.com/airbytehq/airbyte/actions/runs/1999805755
Python tests coverage:

Name Stmts Miss Cover ------------------------------------------------------------------------ source_acceptance_test/utils/__init__.py 6 0 100% source_acceptance_test/tests/__init__.py 4 0 100% source_acceptance_test/__init__.py 2 0 100% source_acceptance_test/tests/test_full_refresh.py 52 2 96% source_acceptance_test/utils/asserts.py 37 2 95% source_acceptance_test/config.py 74 6 92% source_acceptance_test/utils/json_schema_helper.py 105 13 88% source_acceptance_test/utils/common.py 70 17 76% source_acceptance_test/utils/compare.py 62 23 63% source_acceptance_test/tests/test_core.py 275 106 61% source_acceptance_test/base.py 10 4 60% source_acceptance_test/utils/connector_runner.py 110 48 56% source_acceptance_test/tests/test_incremental.py 69 38 45% ------------------------------------------------------------------------ TOTAL 876 259 70% Name Stmts Miss Cover -------------------------------------------------------- source_zendesk_support/__init__.py 2 0 100% source_zendesk_support/streams.py 290 98 66% source_zendesk_support/source.py 45 25 44% -------------------------------------------------------- TOTAL 337 123 64%

bazarnov · 2022-03-18T11:56:20Z

/test connector=connectors/source-zendesk-support

🕑 connectors/source-zendesk-support https://github.com/airbytehq/airbyte/actions/runs/2003958943
✅ connectors/source-zendesk-support https://github.com/airbytehq/airbyte/actions/runs/2003958943
Python tests coverage:

Name Stmts Miss Cover ------------------------------------------------------------------------ source_acceptance_test/utils/__init__.py 6 0 100% source_acceptance_test/tests/__init__.py 4 0 100% source_acceptance_test/__init__.py 2 0 100% source_acceptance_test/tests/test_full_refresh.py 52 2 96% source_acceptance_test/utils/asserts.py 37 2 95% source_acceptance_test/config.py 74 6 92% source_acceptance_test/utils/json_schema_helper.py 105 13 88% source_acceptance_test/utils/common.py 70 17 76% source_acceptance_test/utils/compare.py 62 23 63% source_acceptance_test/tests/test_core.py 275 106 61% source_acceptance_test/base.py 10 4 60% source_acceptance_test/utils/connector_runner.py 110 48 56% source_acceptance_test/tests/test_incremental.py 69 38 45% ------------------------------------------------------------------------ TOTAL 876 259 70% Name Stmts Miss Cover -------------------------------------------------------- source_zendesk_support/__init__.py 2 0 100% source_zendesk_support/streams.py 300 101 66% source_zendesk_support/source.py 45 25 44% -------------------------------------------------------- TOTAL 347 126 64%

bazarnov · 2022-03-18T12:51:30Z

/test connector=connectors/source-zendesk-support

🕑 connectors/source-zendesk-support https://github.com/airbytehq/airbyte/actions/runs/2004201618
✅ connectors/source-zendesk-support https://github.com/airbytehq/airbyte/actions/runs/2004201618
Python tests coverage:

Name Stmts Miss Cover ------------------------------------------------------------------------ source_acceptance_test/utils/__init__.py 6 0 100% source_acceptance_test/tests/__init__.py 4 0 100% source_acceptance_test/__init__.py 2 0 100% source_acceptance_test/tests/test_full_refresh.py 52 2 96% source_acceptance_test/utils/asserts.py 37 2 95% source_acceptance_test/config.py 74 6 92% source_acceptance_test/utils/json_schema_helper.py 105 13 88% source_acceptance_test/utils/common.py 70 17 76% source_acceptance_test/utils/compare.py 62 23 63% source_acceptance_test/tests/test_core.py 275 106 61% source_acceptance_test/base.py 10 4 60% source_acceptance_test/utils/connector_runner.py 110 48 56% source_acceptance_test/tests/test_incremental.py 69 38 45% ------------------------------------------------------------------------ TOTAL 876 259 70% Name Stmts Miss Cover -------------------------------------------------------- source_zendesk_support/__init__.py 2 0 100% source_zendesk_support/streams.py 294 96 67% source_zendesk_support/source.py 45 25 44% -------------------------------------------------------- TOTAL 341 121 65%

sherifnada

some small change requests

docs/integrations/sources/zendesk-support.md

airbyte-integrations/connectors/source-zendesk-support/source_zendesk_support/streams.py

bazarnov · 2022-03-19T01:25:55Z

/test connector=connectors/source-zendesk-support

🕑 connectors/source-zendesk-support https://github.com/airbytehq/airbyte/actions/runs/2007186683
✅ connectors/source-zendesk-support https://github.com/airbytehq/airbyte/actions/runs/2007186683
Python tests coverage:

Name Stmts Miss Cover ------------------------------------------------------------------------ source_acceptance_test/utils/__init__.py 6 0 100% source_acceptance_test/tests/__init__.py 4 0 100% source_acceptance_test/__init__.py 2 0 100% source_acceptance_test/tests/test_full_refresh.py 52 2 96% source_acceptance_test/utils/asserts.py 37 2 95% source_acceptance_test/config.py 74 6 92% source_acceptance_test/utils/json_schema_helper.py 105 13 88% source_acceptance_test/utils/common.py 70 17 76% source_acceptance_test/utils/compare.py 62 23 63% source_acceptance_test/tests/test_core.py 275 106 61% source_acceptance_test/base.py 10 4 60% source_acceptance_test/utils/connector_runner.py 110 48 56% source_acceptance_test/tests/test_incremental.py 69 38 45% ------------------------------------------------------------------------ TOTAL 876 259 70% Name Stmts Miss Cover -------------------------------------------------------- source_zendesk_support/__init__.py 2 0 100% source_zendesk_support/streams.py 284 81 71% source_zendesk_support/source.py 45 25 44% -------------------------------------------------------- TOTAL 331 106 68%

sherifnada

Thanks for adding those unit tests!~ Just a couple more requested and we should be good to go

airbyte-integrations/connectors/source-zendesk-support/source_zendesk_support/streams.py

bazarnov · 2022-03-20T13:20:08Z

/test connector=connectors/source-zendesk-support

🕑 connectors/source-zendesk-support https://github.com/airbytehq/airbyte/actions/runs/2011922667
✅ connectors/source-zendesk-support https://github.com/airbytehq/airbyte/actions/runs/2011922667
Python tests coverage:

Name Stmts Miss Cover ------------------------------------------------------------------------ source_acceptance_test/utils/__init__.py 6 0 100% source_acceptance_test/tests/__init__.py 4 0 100% source_acceptance_test/__init__.py 2 0 100% source_acceptance_test/tests/test_full_refresh.py 52 2 96% source_acceptance_test/utils/asserts.py 37 2 95% source_acceptance_test/config.py 74 6 92% source_acceptance_test/utils/json_schema_helper.py 105 13 88% source_acceptance_test/utils/common.py 70 17 76% source_acceptance_test/utils/compare.py 62 23 63% source_acceptance_test/tests/test_core.py 275 106 61% source_acceptance_test/base.py 10 4 60% source_acceptance_test/utils/connector_runner.py 110 48 56% source_acceptance_test/tests/test_incremental.py 69 38 45% ------------------------------------------------------------------------ TOTAL 876 259 70% Name Stmts Miss Cover -------------------------------------------------------- source_zendesk_support/__init__.py 2 0 100% source_zendesk_support/streams.py 284 57 80% source_zendesk_support/source.py 45 25 44% -------------------------------------------------------- TOTAL 331 82 75%

airbyte-integrations/connectors/source-zendesk-support/unit_tests/test_other.py

bazarnov · 2022-03-21T10:18:49Z

/publish connector=connectors/source-zendesk-support

🕑 connectors/source-zendesk-support https://github.com/airbytehq/airbyte/actions/runs/2015437734
✅ connectors/source-zendesk-support https://github.com/airbytehq/airbyte/actions/runs/2015437734

Fixed the bug when TicketComments stream didn't return all records

a2c4b8c

bazarnov self-assigned this Mar 17, 2022

github-actions bot added the area/connectors Connector related issues label Mar 17, 2022

bazarnov linked an issue Mar 17, 2022 that may be closed by this pull request

🐛 Source Zendesk Support: [ticket comment stream] stream doesn't pull all comments #8519

Closed

added changelog

8072c5e

github-actions bot added the area/documentation Improvements or additions to documentation label Mar 17, 2022

bazarnov requested review from annalvova05 and lazebnyi March 17, 2022 16:18

bazarnov temporarily deployed to more-secrets March 17, 2022 16:20 Inactive

bazarnov temporarily deployed to more-secrets March 17, 2022 16:21 Inactive

formated

0da8663

bazarnov temporarily deployed to more-secrets March 17, 2022 16:27 Inactive

fixed flakeCheck

bc5e314

bazarnov temporarily deployed to more-secrets March 17, 2022 16:32 Inactive

moved the logic to ZendeskSupportTicketEventsStream class

46576cb

bazarnov temporarily deployed to more-secrets March 18, 2022 11:58 Inactive

Oleksandr Bazarnov added 2 commits March 18, 2022 14:34

fixed the documentation

70b8c99

fixed the documentation

a1f6a96

bazarnov temporarily deployed to more-secrets March 18, 2022 12:37 Inactive

removed filter_by_state method as not needed

6d3e3e4

bazarnov temporarily deployed to more-secrets March 18, 2022 12:52 Inactive

bazarnov requested a review from sherifnada March 18, 2022 17:38

sherifnada suggested changes Mar 18, 2022

View reviewed changes

updated after review

b3bf785

bazarnov requested a review from sherifnada March 19, 2022 01:26

bazarnov temporarily deployed to more-secrets March 19, 2022 01:28 Inactive

sherifnada reviewed Mar 19, 2022

View reviewed changes

updated unit_tests based on review

94c1310

bazarnov requested a review from sherifnada March 20, 2022 13:20

bazarnov temporarily deployed to more-secrets March 20, 2022 13:22 Inactive

sherifnada approved these changes Mar 21, 2022

View reviewed changes

airbyte-integrations/connectors/source-zendesk-support/unit_tests/test_other.py Show resolved Hide resolved

Oleksandr Bazarnov added 2 commits March 21, 2022 12:17

changed the name of the unit_test file

4ef4aa8

changed the name of the unit_test file

91a3673

bazarnov temporarily deployed to more-secrets March 21, 2022 10:21 Inactive

new spec

a2bc55c

bazarnov temporarily deployed to more-secrets March 21, 2022 11:34 Inactive

bazarnov merged commit 94a862b into master Mar 21, 2022

bazarnov deleted the bazarnov/source-zendesk-support-fix-ticket-comments branch March 21, 2022 11:51

octavia-squidington-iii mentioned this pull request Mar 21, 2022

Bump Airbyte version from 0.35.58-alpha to 0.35.59-alpha #11298

Merged

girarda mentioned this pull request Apr 2, 2022

Source Zendesk Support: Delete unused field and update links to dec #11671

Merged

lazebnyi removed their request for review May 30, 2022 10:57

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

🐛 Source Zendesk Support: refactor TicketComments stream #11237

🐛 Source Zendesk Support: refactor TicketComments stream #11237

Uh oh!

bazarnov commented Mar 17, 2022 •

edited

Loading

codecov bot commented Mar 17, 2022 •

edited

Loading

bazarnov commented Mar 17, 2022 •

edited by github-actions bot

Loading

bazarnov commented Mar 18, 2022 •

edited by github-actions bot

Loading

bazarnov commented Mar 18, 2022 •

edited by github-actions bot

Loading

sherifnada left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bazarnov commented Mar 19, 2022 •

edited by github-actions bot

Loading

sherifnada left a comment

Uh oh!

Uh oh!

Uh oh!

bazarnov commented Mar 20, 2022 •

edited by github-actions bot

Loading

Uh oh!

bazarnov commented Mar 21, 2022 •

edited by github-actions bot

Loading

Labels

3 participants

🐛 Source Zendesk Support: refactor TicketComments stream #11237

🐛 Source Zendesk Support: refactor TicketComments stream #11237

Uh oh!

Conversation

bazarnov commented Mar 17, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

How

🚨 User Impact 🚨

Pre-merge Checklist

Community member or Airbyter

Airbyter

codecov bot commented Mar 17, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

bazarnov commented Mar 17, 2022 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

bazarnov commented Mar 18, 2022 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

bazarnov commented Mar 18, 2022 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

sherifnada left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bazarnov commented Mar 19, 2022 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

sherifnada left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

bazarnov commented Mar 20, 2022 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bazarnov commented Mar 21, 2022 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Labels

3 participants

bazarnov commented Mar 17, 2022 •

edited

Loading

codecov bot commented Mar 17, 2022 •

edited

Loading

bazarnov commented Mar 17, 2022 •

edited by github-actions bot

Loading

bazarnov commented Mar 18, 2022 •

edited by github-actions bot

Loading

bazarnov commented Mar 18, 2022 •

edited by github-actions bot

Loading

bazarnov commented Mar 19, 2022 •

edited by github-actions bot

Loading

bazarnov commented Mar 20, 2022 •

edited by github-actions bot

Loading

bazarnov commented Mar 21, 2022 •

edited by github-actions bot

Loading