Source Hubspot: new stream Email Subscriptions #22910

aazam-gh · 2023-02-13T13:15:51Z

What

Adding a new stream email subscriptions
Returns all email subscription types that have been created in the given Hub ID.

How

The connector adds the email subscription API stream
Fixes #17034

🚨 User Impact 🚨

No changes to existing code

Pre-merge Checklist

Expand the relevant checklist and delete the others.

Updating a connector

Community member or Airbyter

Grant edit access to maintainers (instructions)
Secrets in the connector's spec are annotated with airbyte_secret
[] Unit & integration tests added and passing. Community members, please provide proof of success locally e.g: screenshot or copy-paste unit, integration, and acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
[] Code reviews completed
Documentation updated
- Connector's README.md
- Connector's bootstrap.md. See description and examples
- Changelog updated in docs/integrations/<source or destination>/<name>.md including changelog. See changelog example
PR name follows PR naming conventions

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

Create a non-forked branch based on this PR and test the below items on it
Build is successful
If new credentials are required for use in CI, add them to GSM. Instructions.
/test connector=connectors/<name> command is passing
New Connector version released on Dockerhub and connector version bumped by running the /publish command described here

aazam-gh · 2023-02-13T13:17:04Z

The integration test for incremental read fails. The stream is not an incremental one

YowanR · 2023-02-17T22:28:44Z

@Alcadeus0 Is it possible to make this stream incremental? Is there a cursor available?

aazam-gh · 2023-02-18T03:23:40Z

@YowanR Unfortunately this stream doesn't have a cursor field available

YowanR · 2023-02-27T22:11:25Z

@sh4sh @natalyjazzviolin Can you please take a look at this PR? Thanks 🙏

YowanR · 2023-03-17T15:16:56Z

@marcosmarxm @sajarin can you take a look?

marcosmarxm · 2023-03-17T17:24:06Z

/test connector=connectors/source-hubspot

🕑 connectors/source-hubspot https://github.com/airbytehq/airbyte/actions/runs/4450038178
❌ connectors/source-hubspot https://github.com/airbytehq/airbyte/actions/runs/4450038178
🐛 https://gradle.com/s/mij3hzz3dw5uo

Build Failed

Test summary info:

 =========================== short test summary info ============================ ERROR integration_tests/test_associations.py::test_incremental_read_fetches_associations �[31m======================== �[33m55 warnings�[0m, �[31m�[1m1 error�[0m�[31m in 0.99s�[0m�[31m =========================�[0m

merging to resolve conflict

aazam-gh · 2023-03-18T11:35:49Z

@marcosmarxm can you take a look

marcosmarxm · 2023-03-21T16:26:08Z

/test connector=connectors/source-hubspot

🕑 connectors/source-hubspot https://github.com/airbytehq/airbyte/actions/runs/4481449916
❌ connectors/source-hubspot https://github.com/airbytehq/airbyte/actions/runs/4481449916
🐛 https://gradle.com/s/mvdx7lujkb546

Build Failed

Test summary info:

=========================== short test summary info ============================ FAILED test_core.py::TestDiscovery::test_primary_keys_exist_in_schema[inputs0] ERROR test_core.py::TestBasicRead::test_read[inputs0] - Failed: High strictne... SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:100: The previous and actual specifications are identical. ========= 1 failed, 36 passed, 1 skipped, 1 error in 415.19s (0:06:55) =========

aazam-gh · 2023-03-21T16:36:35Z

/test connector=connectors/source-hubspot

🕑 connectors/source-hubspot https://github.com/airbytehq/airbyte/actions/runs/4481449916

What does that mean?

marcosmarxm

Left some comments about the stream schema please fix them.

marcosmarxm · 2023-03-22T16:14:29Z

airbyte-integrations/connectors/source-hubspot/source_hubspot/schemas/email_subscriptions.json

+{
+ "$schema": "http://json-schema.org/draft-07/schema#",
+ "type": ["null", "object"],
+ "properties": {


missing the additionalProperties field, please check others stream.

The other streams don't have the additional properties field as well. Can you point me to the docs where they are?

marcosmarxm · 2023-03-22T16:16:34Z

airbyte-integrations/connectors/source-hubspot/source_hubspot/schemas/email_subscriptions.json

+ "$schema": "http://json-schema.org/draft-07/schema#",
+ "type": ["null", "object"],
+ "properties": {
+ "subscriptionDefinitions": {


The schema is wrong. You want an unity record which is the data inside the subscriptionDefinitions

You're right, I've removed the subscription definition and made it point as a unitary one

marcosmarxm · 2023-03-31T21:58:35Z

/test connector=source-hubspot

🕑 source-hubspot https://github.com/airbytehq/airbyte/actions/runs/4579476271
✅ source-hubspot https://github.com/airbytehq/airbyte/actions/runs/4579476271
Python tests coverage:

Name Stmts Miss Cover ------------------------------------------------- source_hubspot/errors.py 6 0 100% source_hubspot/constants.py 2 0 100% source_hubspot/__init__.py 2 0 100% source_hubspot/helpers.py 70 3 96% source_hubspot/streams.py 841 77 91% source_hubspot/source.py 60 19 68% ------------------------------------------------- TOTAL 981 99 90% Name Stmts Miss Cover ------------------------------------------------- source_hubspot/errors.py 6 0 100% source_hubspot/constants.py 2 0 100% source_hubspot/__init__.py 2 0 100% source_hubspot/helpers.py 70 10 86% source_hubspot/source.py 60 14 77% source_hubspot/streams.py 841 217 74% ------------------------------------------------- TOTAL 981 241 75%

Build Passed

Test summary info:

=========================== short test summary info ============================ SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:100: The previous and actual specifications are identical. ================== 41 passed, 1 skipped in 408.18s (0:06:48) ===================

aazam-gh · 2023-03-31T23:42:28Z

@marcosmarxm Are the tests not successful? It should have merged right?

marcosmarxm · 2023-04-01T01:40:43Z

@marcosmarxm Are the tests not successful? It should have merged right?

Yes, Tests are working now. I need the final approval from the connector team and it is ready to merge. There isn't anything in your side right now! Thanks for the contribution @Alcadeus0

marcosmarxm

Thanks @Alcadeus0

aazam-gh · 2023-04-01T04:27:55Z

Thanks for your help with the review :)

lazebnyi

Nice!

marcosmarxm · 2023-04-03T14:54:16Z

/publish connector=connectors/source-hubspot

🕑 Publishing the following connectors:
connectors/source-hubspot
https://github.com/airbytehq/airbyte/actions/runs/4598148655

Connector	Did it publish?	Were definitions generated?
connectors/source-hubspot	✅	✅

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

aazam-gh added 2 commits February 11, 2023 19:44

Check connection works

5e5fbbc

Fixed reading records and passing unit tests

0f0b319

octavia-squidington-iii added area/connectors Connector related issues bounty community connectors/source/hubspot labels Feb 13, 2023

Added stream in oauth and full_refresh catalogs

24326bc

YowanR requested review from natalyjazzviolin and sh4sh February 27, 2023 22:11

alexopenline mentioned this pull request Mar 8, 2023

Add Hubspot calendar events to timeline customeros/customeros#1239

Closed

YowanR requested a review from a team March 17, 2023 15:16

aazam-gh and others added 5 commits March 18, 2023 12:12

Merge branch 'airbytehq:master' into emailsubapi

72f4561

Check connection works

83cb520

Fixed reading records and passing unit tests

717860f

Added stream in oauth and full_refresh catalogs

917ed95

Merge remote-tracking branch 'origin/emailsubapi' into emailsubapi

1695faf

merging to resolve conflict

Merge branch 'master' into emailsubapi

9d13180

marcosmarxm added the contributor-program PRs submitted through the contributor program. label Mar 22, 2023

marcosmarxm suggested changes Mar 22, 2023

View reviewed changes

aazam-gh added 2 commits March 24, 2023 01:14

updated schema

47eabe4

updated docs

6d536c2

octavia-squidington-iii added the area/documentation Improvements or additions to documentation label Mar 28, 2023

aazam-gh and others added 4 commits March 28, 2023 20:07

updated streams.py

8b43a60

add additionalProperties to email subscriptions

cec3784

Merge branch 'master' into emailsubapi

b63f3d1

add expected records

92a4d87

marcosmarxm requested review from lazebnyi and removed request for natalyjazzviolin April 1, 2023 01:41

marcosmarxm approved these changes Apr 1, 2023

View reviewed changes

lazebnyi approved these changes Apr 3, 2023

View reviewed changes

Merge branch 'master' into emailsubapi

03b3672

auto-bump connector version

e817321

marcosmarxm merged commit fcfcf49 into airbytehq:master Apr 3, 2023

aazam-gh deleted the emailsubapi branch June 7, 2023 16:55

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Source Hubspot: new stream Email Subscriptions #22910

Source Hubspot: new stream Email Subscriptions #22910

Uh oh!

aazam-gh commented Feb 13, 2023 •

edited

Loading

aazam-gh commented Feb 13, 2023

YowanR commented Feb 17, 2023

aazam-gh commented Feb 18, 2023

YowanR commented Feb 27, 2023

YowanR commented Mar 17, 2023

marcosmarxm commented Mar 17, 2023 •

edited by github-actions bot

Loading

aazam-gh commented Mar 18, 2023 •

edited

Loading

marcosmarxm commented Mar 21, 2023 •

edited by github-actions bot

Loading

aazam-gh commented Mar 21, 2023

marcosmarxm left a comment

marcosmarxm Mar 22, 2023

aazam-gh Mar 23, 2023

marcosmarxm Mar 22, 2023

aazam-gh Mar 23, 2023

marcosmarxm commented Mar 31, 2023 •

edited by github-actions bot

Loading

aazam-gh commented Mar 31, 2023

marcosmarxm commented Apr 1, 2023

marcosmarxm left a comment

aazam-gh commented Apr 1, 2023

lazebnyi left a comment

marcosmarxm commented Apr 3, 2023 •

edited by github-actions bot

Loading

Labels

5 participants

Source Hubspot: new stream Email Subscriptions #22910

Source Hubspot: new stream Email Subscriptions #22910

Uh oh!

Conversation

aazam-gh commented Feb 13, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

How

Recommended reading order

🚨 User Impact 🚨

Pre-merge Checklist

Community member or Airbyter

Airbyter

aazam-gh commented Feb 13, 2023

YowanR commented Feb 17, 2023

aazam-gh commented Feb 18, 2023

YowanR commented Feb 27, 2023

YowanR commented Mar 17, 2023

marcosmarxm commented Mar 17, 2023 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Build Failed

aazam-gh commented Mar 18, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

marcosmarxm commented Mar 21, 2023 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Build Failed

aazam-gh commented Mar 21, 2023

marcosmarxm left a comment

Choose a reason for hiding this comment

marcosmarxm Mar 22, 2023

Choose a reason for hiding this comment

aazam-gh Mar 23, 2023

Choose a reason for hiding this comment

marcosmarxm Mar 22, 2023

Choose a reason for hiding this comment

aazam-gh Mar 23, 2023

Choose a reason for hiding this comment

marcosmarxm commented Mar 31, 2023 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Build Passed

aazam-gh commented Mar 31, 2023

marcosmarxm commented Apr 1, 2023

marcosmarxm left a comment

Choose a reason for hiding this comment

aazam-gh commented Apr 1, 2023

lazebnyi left a comment

Choose a reason for hiding this comment

marcosmarxm commented Apr 3, 2023 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Labels

5 participants

aazam-gh commented Feb 13, 2023 •

edited

Loading

marcosmarxm commented Mar 17, 2023 •

edited by github-actions bot

Loading

aazam-gh commented Mar 18, 2023 •

edited

Loading

marcosmarxm commented Mar 21, 2023 •

edited by github-actions bot

Loading

marcosmarxm commented Mar 31, 2023 •

edited by github-actions bot

Loading

marcosmarxm commented Apr 3, 2023 •

edited by github-actions bot

Loading