🎉 Source Google Ads: add *_label streams #11221

edgao · 2022-03-17T03:29:54Z

What

These streams tie together the label IDs with their actual names; users can sync these streams and then join them downstream. GAQL makes it a real pain to do that join (we'd have to do it ourselves in Python because they don't attribute the *_label resources to their corresponding tables).

For example, the ad_groups stream might output records like {"ad_group.labels":["customers/4651612872/labels/21585034471"], ....}. ad_group_labels has records that look like this:

{ "ad_group.resource_name": "customers/4651612872/adGroups/123273719655", "ad_group_label.resource_name": "customers/4651612872/adGroupLabels/123273719655~21585034471", "label.name": "edgao-example-label", "label.resource_name": "customers/4651612872/labels/21585034471" }

Users would be expected to join these two streams together in order to use attach label.name field to the ad_group records.

🚨 User Impact 🚨

nope

Pre-merge Checklist

Expand the relevant checklist and delete the others.

Community member or Airbyter

Grant edit access to maintainers (instructions)
Secrets in the connector's spec are annotated with airbyte_secret
Unit & integration tests added and passing. Community members, please provide proof of success locally e.g: screenshot or copy-paste unit, integration, and acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
Code reviews completed
Documentation updated
- Connector's README.md
- Connector's bootstrap.md. See description and examples
- Changelog updated in docs/integrations/<source or destination>/<name>.md including changelog. See changelog example
PR name follows PR naming conventions

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

Create a non-forked branch based on this PR and test the below items on it
Build is successful
If new credentials are required for use in CI, add them to GSM. Instructions.
/test connector=connectors/<name> command is passing
New Connector version released on Dockerhub by running the /publish command described here
After the new connector version is published, connector version bumped in the seed directory as described here
Seed specs have been re-generated by building the platform and committing the changes to the seed spec files, as described here

edgao · 2022-03-18T22:00:30Z

/test connector=connectors/source-google-ads

edgao · 2022-03-21T03:04:00Z

/test connector=connectors/source-google-ads

🕑 connectors/source-google-ads https://github.com/airbytehq/airbyte/actions/runs/2013896002
✅ connectors/source-google-ads https://github.com/airbytehq/airbyte/actions/runs/2013896002
Python tests coverage:

Name Stmts Miss Cover ------------------------------------------------------------------------ source_acceptance_test/utils/__init__.py 6 0 100% source_acceptance_test/tests/__init__.py 4 0 100% source_acceptance_test/__init__.py 2 0 100% source_acceptance_test/tests/test_full_refresh.py 52 2 96% source_acceptance_test/utils/asserts.py 37 2 95% source_acceptance_test/config.py 74 6 92% source_acceptance_test/utils/json_schema_helper.py 105 13 88% source_acceptance_test/utils/common.py 70 17 76% source_acceptance_test/utils/compare.py 62 23 63% source_acceptance_test/tests/test_core.py 275 106 61% source_acceptance_test/base.py 10 4 60% source_acceptance_test/utils/connector_runner.py 110 48 56% source_acceptance_test/tests/test_incremental.py 69 38 45% ------------------------------------------------------------------------ TOTAL 876 259 70% Name Stmts Miss Cover -------------------------------------------------------------- source_google_ads/__init__.py 2 0 100% source_google_ads/google_ads.py 68 10 85% source_google_ads/streams.py 142 24 83% source_google_ads/source.py 73 24 67% source_google_ads/custom_query_stream.py 75 50 33% -------------------------------------------------------------- TOTAL 360 108 70% Name Stmts Miss Cover -------------------------------------------------------------- source_google_ads/__init__.py 2 0 100% source_google_ads/streams.py 142 11 92% source_google_ads/custom_query_stream.py 75 6 92% source_google_ads/google_ads.py 68 7 90% source_google_ads/source.py 73 19 74% -------------------------------------------------------------- TOTAL 360 43 88%

Python short test summary info:

=========================== short test summary info ============================ SKIPPED [1] ../usr/local/lib/python3.7/site-packages/source_acceptance_test/plugin.py:56: Skipping TestIncremental.test_two_sequential_reads because not found in the config ================== 21 passed, 1 skipped in 1331.02s (0:22:11) ==================

edgao · 2022-03-22T20:59:27Z

/test connector=connectors/source-google-ads

edgao · 2022-03-22T21:03:12Z

/test connector=connectors/source-google-ads

🕑 connectors/source-google-ads https://github.com/airbytehq/airbyte/actions/runs/2024863586
✅ connectors/source-google-ads https://github.com/airbytehq/airbyte/actions/runs/2024863586
Python tests coverage:

Name Stmts Miss Cover ------------------------------------------------------------------------ source_acceptance_test/utils/__init__.py 6 0 100% source_acceptance_test/tests/__init__.py 4 0 100% source_acceptance_test/__init__.py 2 0 100% source_acceptance_test/tests/test_full_refresh.py 52 2 96% source_acceptance_test/utils/asserts.py 37 2 95% source_acceptance_test/config.py 74 6 92% source_acceptance_test/utils/json_schema_helper.py 105 13 88% source_acceptance_test/utils/common.py 70 17 76% source_acceptance_test/utils/compare.py 62 23 63% source_acceptance_test/tests/test_core.py 275 106 61% source_acceptance_test/base.py 10 4 60% source_acceptance_test/utils/connector_runner.py 110 48 56% source_acceptance_test/tests/test_incremental.py 69 38 45% ------------------------------------------------------------------------ TOTAL 876 259 70% Name Stmts Miss Cover -------------------------------------------------------------- source_google_ads/__init__.py 2 0 100% source_google_ads/google_ads.py 68 10 85% source_google_ads/streams.py 142 24 83% source_google_ads/source.py 73 24 67% source_google_ads/custom_query_stream.py 75 50 33% -------------------------------------------------------------- TOTAL 360 108 70% Name Stmts Miss Cover -------------------------------------------------------------- source_google_ads/__init__.py 2 0 100% source_google_ads/streams.py 142 11 92% source_google_ads/custom_query_stream.py 75 6 92% source_google_ads/google_ads.py 68 7 90% source_google_ads/source.py 73 19 74% -------------------------------------------------------------- TOTAL 360 43 88%

Python short test summary info:

=========================== short test summary info ============================ SKIPPED [1] ../usr/local/lib/python3.7/site-packages/source_acceptance_test/plugin.py:56: Skipping TestIncremental.test_two_sequential_reads because not found in the config ================== 21 passed, 1 skipped in 1633.21s (0:27:13) ==================

edgao · 2022-03-22T22:16:58Z

docs/integrations/sources/google-ads.md

 * [keyword](https://developers.google.com/google-ads/api/fields/v8/keyword_view)
 * [geographic](https://developers.google.com/google-ads/api/fields/v8/geographic_view)

+Note that `ad_groups`, `ad_group_ads`, and `campaigns` contain a `labels` field, which should be joined against their respective `*_labels` streams if you want to view the actual labels.


@misteryeo does this sentence sound good to you? (this is to address #10662 )

could you indicate the join field?

sherifnada · 2022-03-22T22:23:52Z

docs/integrations/sources/google-ads.md

 * [keyword](https://developers.google.com/google-ads/api/fields/v8/keyword_view)
 * [geographic](https://developers.google.com/google-ads/api/fields/v8/geographic_view)

+Note that `ad_groups`, `ad_group_ads`, and `campaigns` contain a `labels` field, which should be joined against their respective `*_labels` streams if you want to view the actual labels.


could you indicate the join field?

airbyte-integrations/connectors/source-google-ads/source_google_ads/streams.py

edgao · 2022-03-23T15:34:09Z

/publish connector=connectors/source-google-ads

edgao · 2022-03-23T18:30:07Z

/publish connector=connectors/source-google-ads

🕑 connectors/source-google-ads https://github.com/airbytehq/airbyte/actions/runs/2030017453
✅ connectors/source-google-ads https://github.com/airbytehq/airbyte/actions/runs/2030017453

github-actions bot added the area/connectors Connector related issues label Mar 17, 2022

edgao added 3 commits March 18, 2022 13:51

add campaign label stream

bb6db95

use correct fields

ab2eb8f

add ad_group and ad_group_ad label streams

92db4e5

edgao force-pushed the edgao/source_gads_campaign_label branch from 8e3d050 to 92db4e5 Compare March 18, 2022 21:11

edgao added 2 commits March 21, 2022 13:31

ordering + fields

2a9303c

also fetch label.resource_name; add comment

4ef57db

add to connector doc

036821d

github-actions bot added the area/documentation Improvements or additions to documentation label Mar 22, 2022

edgao commented Mar 22, 2022

View reviewed changes

edgao marked this pull request as ready for review March 22, 2022 22:17

edgao requested a review from sherifnada March 22, 2022 22:18

sherifnada approved these changes Mar 22, 2022

View reviewed changes

edgao added 4 commits March 22, 2022 16:43

add join field explanation

10c93a2

remove unnecessary constructor

32b8714

Merge branch 'master' into edgao/source_gads_campaign_label

40ef6d9

bump version; changelog

b585609

regenerate

697784e

edgao temporarily deployed to more-secrets March 23, 2022 20:36 Inactive

edgao temporarily deployed to more-secrets March 23, 2022 20:37 Inactive

edgao merged commit d61af1b into master Mar 23, 2022

edgao deleted the edgao/source_gads_campaign_label branch March 23, 2022 20:37

This was referenced Mar 23, 2022

Bump Airbyte version from 0.35.59-alpha to 0.35.60-alpha #11373

Closed

Bump Airbyte version from 0.35.59-alpha to 0.35.60-alpha #11402

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

🎉 Source Google Ads: add *_label streams #11221

🎉 Source Google Ads: add *_label streams #11221

Uh oh!

edgao commented Mar 17, 2022 •

edited

Loading

edgao commented Mar 18, 2022

edgao commented Mar 21, 2022 •

edited by github-actions bot

Loading

edgao commented Mar 22, 2022

edgao commented Mar 22, 2022 •

edited by github-actions bot

Loading

edgao Mar 22, 2022

sherifnada Mar 22, 2022

edgao Mar 22, 2022

sherifnada Mar 22, 2022

Uh oh!

edgao commented Mar 23, 2022

edgao commented Mar 23, 2022 •

edited by github-actions bot

Loading

Labels

3 participants

🎉 Source Google Ads: add *_label streams #11221

🎉 Source Google Ads: add *_label streams #11221

Uh oh!

Conversation

edgao commented Mar 17, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Recommended reading order

🚨 User Impact 🚨

Pre-merge Checklist

Community member or Airbyter

Airbyter

edgao commented Mar 18, 2022

edgao commented Mar 21, 2022 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

edgao commented Mar 22, 2022

edgao commented Mar 22, 2022 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

edgao Mar 22, 2022

Choose a reason for hiding this comment

sherifnada Mar 22, 2022

Choose a reason for hiding this comment

edgao Mar 22, 2022

Choose a reason for hiding this comment

sherifnada Mar 22, 2022

Choose a reason for hiding this comment

Uh oh!

edgao commented Mar 23, 2022

edgao commented Mar 23, 2022 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Labels

3 participants

edgao commented Mar 17, 2022 •

edited

Loading

edgao commented Mar 21, 2022 •

edited by github-actions bot

Loading

edgao commented Mar 22, 2022 •

edited by github-actions bot

Loading

edgao commented Mar 23, 2022 •

edited by github-actions bot

Loading