Source GAv4: Connection check succeeds but sync fails #6657
Description
Current Behavior
Using the latest GA v4 source connector, I setup a connection (using the credentials in lastpass) to connect to view ID 211669975 using Oauth. The connection check succeeded.
However, once the sync ran, it failed with the following error:
2021-10-02 22:49:52 INFO 2021-10-02 22:49:52 �[32mINFO�[m i.a.i.d.b.BigQueryUtils(createTable):112 - {} - Table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=sherif_test, tableId=_airbyte_tmp_aoi_pages}} created successfully 2021-10-02 22:49:52 ERROR Traceback (most recent call last): 2021-10-02 22:49:52 INFO Starting syncing SourceGoogleAnalyticsV4 2021-10-02 22:49:52 ERROR File "/airbyte/integration_code/main.py", line 13, in <module> 2021-10-02 22:49:52 INFO Syncing stream: daily_active_users 2021-10-02 22:49:52 ERROR launch(source, sys.argv[1:]) 2021-10-02 22:49:52 ERROR File "/usr/local/lib/python3.7/site-packages/airbyte_cdk/entrypoint.py", line 105, in launch 2021-10-02 22:49:52 ERROR for message in source_entrypoint.run(parsed_args): 2021-10-02 22:49:52 ERROR File "/usr/local/lib/python3.7/site-packages/airbyte_cdk/entrypoint.py", line 96, in run 2021-10-02 22:49:52 ERROR for message in generator: 2021-10-02 22:49:52 ERROR File "/usr/local/lib/python3.7/site-packages/airbyte_cdk/sources/abstract_source.py", line 108, in read 2021-10-02 22:49:52 ERROR raise e 2021-10-02 22:49:52 ERROR File "/usr/local/lib/python3.7/site-packages/airbyte_cdk/sources/abstract_source.py", line 104, in read 2021-10-02 22:49:52 ERROR internal_config=internal_config, 2021-10-02 22:49:52 ERROR File "/usr/local/lib/python3.7/site-packages/airbyte_cdk/sources/abstract_source.py", line 134, in _read_stream 2021-10-02 22:49:52 ERROR for record in record_iterator: 2021-10-02 22:49:52 ERROR File "/usr/local/lib/python3.7/site-packages/airbyte_cdk/sources/abstract_source.py", line 206, in _read_full_refresh 2021-10-02 22:49:52 ERROR for record in records: 2021-10-02 22:49:52 ERROR File "/usr/local/lib/python3.7/site-packages/airbyte_cdk/sources/streams/http/http.py", line 351, in read_records 2021-10-02 22:49:52 ERROR response = self._send_request(request, request_kwargs) 2021-10-02 22:49:52 ERROR File "/usr/local/lib/python3.7/site-packages/airbyte_cdk/sources/streams/http/http.py", line 318, in _send_request 2021-10-02 22:49:52 ERROR return backoff_handler(user_backoff_handler)(request, request_kwargs) 2021-10-02 22:49:52 ERROR File "/usr/local/lib/python3.7/site-packages/backoff/_sync.py", line 94, in retry 2021-10-02 22:49:52 ERROR ret = target(*args, **kwargs) 2021-10-02 22:49:52 ERROR File "/usr/local/lib/python3.7/site-packages/backoff/_sync.py", line 94, in retry 2021-10-02 22:49:52 ERROR ret = target(*args, **kwargs) 2021-10-02 22:49:52 ERROR File "/usr/local/lib/python3.7/site-packages/airbyte_cdk/sources/streams/http/http.py", line 284, in _send 2021-10-02 22:49:52 ERROR response.raise_for_status() 2021-10-02 22:49:52 ERROR File "/usr/local/lib/python3.7/site-packages/requests/models.py", line 953, in raise_for_status 2021-10-02 22:49:52 ERROR raise HTTPError(http_error_msg, response=self) 2021-10-02 22:49:52 ERROR requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://analyticsreporting.googleapis.com/v4/reports:batchGet 2021-10-02 22:49:52 ERROR Encountered an exception while reading stream SourceGoogleAnalyticsV4 Traceback (most recent call last): File "/usr/local/lib/python3.7/site-packages/airbyte_cdk/sources/abstract_source.py", line 104, in read internal_config=internal_config, File "/usr/local/lib/python3.7/site-packages/airbyte_cdk/sources/abstract_source.py", line 134, in _read_stream for record in record_iterator: File "/usr/local/lib/python3.7/site-packages/airbyte_cdk/sources/abstract_source.py", line 206, in _read_full_refresh for record in records: File "/usr/local/lib/python3.7/site-packages/airbyte_cdk/sources/streams/http/http.py", line 351, in read_records response = self._send_request(request, request_kwargs) File "/usr/local/lib/python3.7/site-packages/airbyte_cdk/sources/streams/http/http.py", line 318, in _send_request return backoff_handler(user_backoff_handler)(request, request_kwargs) File "/usr/local/lib/python3.7/site-packages/backoff/_sync.py", line 94, in retry ret = target(*args, **kwargs) File "/usr/local/lib/python3.7/site-packages/backoff/_sync.py", line 94, in retry ret = target(*args, **kwargs) File "/usr/local/lib/python3.7/site-packages/airbyte_cdk/sources/streams/http/http.py", line 284, in _send response.raise_for_status() File "/usr/local/lib/python3.7/site-packages/requests/models.py", line 953, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://analyticsreporting.googleapis.com/v4/reports:batchGet Logs: logs-1812-0.txt
When I look at the account explorer I also do not have access to the view ID I initially entered (it is a valid view ID but I don't have permissions from the email account I used for oauth)
Expected Behavior
I would have expected the connection check to fail if I don't have access to a view ID.