This repository was archived by the owner on May 17, 2024. It is now read-only.
This repository was archived by the owner on May 17, 2024. It is now read-only.
Unexpected keyword impersonate_service_account error when running a dbt diff with potential fix #758
Description
Describe the bug
I am running a data-diff on a specific model using the following command:
data-diff --dbt -d --select user_metricsWhich yields the following output
Running with data-diff=0.9.8 12:48:00 INFO Parsing file dbt_project.yml dbt_parser.py:287 INFO Parsing file target/manifest.json dbt_parser.py:280 12:48:02 INFO config: prod_database='<REDACTED>' prod_schema='marts' prod_custom_schema='<custom_schema>' datasource_id=None dbt_parser.py:159 INFO Parsing file profiles.yml dbt_parser.py:294 DEBUG Found PKs via META: ['metric_id'] dbt_parser.py:449 ERROR Client.__init__() got an unexpected keyword argument 'impersonate_service_account' Describe the environment
macOS v14.0
data-diff== 0.9.8
dbt-bigquery==1.6.7
dbt-core==1.6.6
** Possible solution **
I had a look at the file ( data_diff/databases/bigquery.py) that was throwing the exception and I think because the impersonate_service_account key is not being popped like the keyfile arg, if a user does not user the impersonate service account functionality (I do not), then when the Big Query client is instantiated, the kw variable contains the unexpected keyword argument
As of 0.9.8 (lines 195-210)
keyfile = kw.pop("keyfile", None) if keyfile: bigquery_service_account = import_bigquery_service_account() credentials = bigquery_service_account.Credentials.from_service_account_file( keyfile, scopes=["https://www.googleapis.com/auth/cloud-platform"], ) elif kw.get("impersonate_service_account"): bigquery_service_account_impersonation = import_bigquery_service_account_impersonation() credentials = bigquery_service_account_impersonation.Credentials( source_credentials=credentials, target_principal=kw["impersonate_service_account"], target_scopes=["https://www.googleapis.com/auth/cloud-platform"], ) self._client = bigquery.Client(project=project, credentials=credentials, **kw)My suggestion
keyfile = kw.pop("keyfile", None) impersonate_service_account = kw.pop("impersonate_service_account", None) if keyfile: bigquery_service_account = import_bigquery_service_account() credentials = bigquery_service_account.Credentials.from_service_account_file( keyfile, scopes=["https://www.googleapis.com/auth/cloud-platform"], ) elif impersonate_service_account: bigquery_service_account_impersonation = import_bigquery_service_account_impersonation() credentials = bigquery_service_account_impersonation.Credentials( source_credentials=credentials, target_principal=impersonate_service_account, target_scopes=["https://www.googleapis.com/auth/cloud-platform"], ) self._client = bigquery.Client(project=project, credentials=credentials, **kw)which has resolved my issue
I am happy to open an MR if needed!