Skip to content

Conversation

@abdelmegahed
Copy link
Contributor

Right now, unsetting ['timePartitioning']['expirationMs'] on BigQuery table is not supported and raises google.api_core.exceptions.BadRequest: 400 PATCH https://bigquery.googleapis.com/bigquery/v2/projects/abdelrahmanm-bd-dev/datasets/partition_expiry_test/tables/sample_data_partitioned?prettyPrint=false: Invalid value at 'table.time_partitioning.expiration_ms.value' (TYPE_INT64), "None"

This happens since the backend doesn't treat "None" (string) as a null. This PR addresses this issue from the client side.

Sample code that fails with the above exception:

from google.cloud import bigquery client = bigquery.Client(project='abdelrahmanm-bd-dev') table_ref = client.dataset('partition_expiry_test').table('sample_data_partitioned') table = client.get_table(table_ref) table.partition_expiration = None # ISSUE HERE!! This get cast to string and the backend returns a 400 as a result. client.update_table(table, ['partition_expiration']) 
@abdelmegahed abdelmegahed requested review from a team as code owners April 17, 2023 20:16
@abdelmegahed abdelmegahed requested a review from rosiezou April 17, 2023 20:16
@product-auto-label product-auto-label bot added size: xs Pull request size is extra small. api: bigquery Issues related to the googleapis/python-bigquery API. labels Apr 17, 2023
Copy link
Contributor

@tswast tswast left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @abdelmegahed

Could you write a unit test covering this case? I'd expect one similar to https://github.com/googleapis/python-bigquery/blob/main/tests/unit/test_table.py#L1178 test_to_api_repr_w_custom_field perhaps called test_to_api_repr_w_unsetting_expiration

@product-auto-label product-auto-label bot added size: s Pull request size is small. and removed size: xs Pull request size is extra small. labels Apr 18, 2023
@abdelmegahed
Copy link
Contributor Author

@tswast test added. Thanks for the pointers :)

@abdelmegahed abdelmegahed requested a review from tswast April 18, 2023 21:54
@tswast tswast added the kokoro:run Add this label to force Kokoro to re-run the tests. label Apr 18, 2023
@yoshi-kokoro yoshi-kokoro removed the kokoro:run Add this label to force Kokoro to re-run the tests. label Apr 18, 2023
@tswast tswast added the kokoro:run Add this label to force Kokoro to re-run the tests. label Apr 18, 2023
@yoshi-kokoro yoshi-kokoro removed the kokoro:run Add this label to force Kokoro to re-run the tests. label Apr 18, 2023
@abdelmegahed abdelmegahed requested a review from tswast April 19, 2023 00:05
@tswast tswast changed the title hotfix: handle case when expirationMs is None fix: handle case when expirationMs is None May 16, 2023
@tswast tswast added kokoro:force-run Add this label to force Kokoro to re-run the tests. owlbot:run Add this label to trigger the Owlbot post processor. labels May 16, 2023
@gcf-owl-bot gcf-owl-bot bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label May 16, 2023
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label May 16, 2023
@tswast tswast merged commit fa6e13d into googleapis:main May 17, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api: bigquery Issues related to the googleapis/python-bigquery API. size: s Pull request size is small.

3 participants