Skip to content

Commit 7162f98

Browse files
tswasttswast
authored
Merge pull request #1049 from googleapis/sync-v3
chore: sync v3 with main branch
2 parents 40c92c3 + 61e3d57 commit 7162f98

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

49 files changed

+1287
-148
lines changed

‎.kokoro/presubmit/prerelease-deps-3.8.cfg‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,5 +3,5 @@
33
# Only run this nox session.
44
env_vars: {
55
key: "NOX_SESSION"
6-
value: "prerelease_deps"
7-
}
6+
value: "prerelease_deps-3.8"
7+
}

‎.kokoro/presubmit/snippets-3.10.cfg‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
# Format: //devtools/kokoro/config/proto/build.proto
2+
3+
# Only run this nox session.
4+
env_vars: {
5+
key: "NOX_SESSION"
6+
value: "snippets-3.10"
7+
}

‎.kokoro/presubmit/snippets-2.7.cfg‎ renamed to ‎.kokoro/presubmit/system-3.10.cfg‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,5 +3,5 @@
33
# Only run this nox session.
44
env_vars: {
55
key: "NOX_SESSION"
6-
value: "snippets-2.7"
6+
value: "system-3.10"
77
}

‎CHANGELOG.md‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,54 @@
55
[1]: https://pypi.org/project/google-cloud-bigquery/#history
66

77

8+
### [2.30.1](https://www.github.com/googleapis/python-bigquery/compare/v2.30.0...v2.30.1) (2021-11-04)
9+
10+
11+
### Bug Fixes
12+
13+
* error if eval()-ing repr(SchemaField) ([#1046](https://www.github.com/googleapis/python-bigquery/issues/1046)) ([13ac860](https://www.github.com/googleapis/python-bigquery/commit/13ac860de689ea13b35932c67042bc35e388cb30))
14+
15+
16+
### Documentation
17+
18+
* show gcloud command to authorize against sheets ([#1045](https://www.github.com/googleapis/python-bigquery/issues/1045)) ([20c9024](https://www.github.com/googleapis/python-bigquery/commit/20c9024b5760f7ae41301f4da54568496922cbe2))
19+
* use stable URL for pandas intersphinx links ([#1048](https://www.github.com/googleapis/python-bigquery/issues/1048)) ([73312f8](https://www.github.com/googleapis/python-bigquery/commit/73312f8f0f22ff9175a4f5f7db9bb438a496c164))
20+
21+
## [2.30.0](https://www.github.com/googleapis/python-bigquery/compare/v2.29.0...v2.30.0) (2021-11-03)
22+
23+
24+
### Features
25+
26+
* accept TableListItem where TableReference is accepted ([#1016](https://www.github.com/googleapis/python-bigquery/issues/1016)) ([fe16adc](https://www.github.com/googleapis/python-bigquery/commit/fe16adc86a170d0992c32091b349b036f8b43884))
27+
* support Python 3.10 ([#1043](https://www.github.com/googleapis/python-bigquery/issues/1043)) ([5bbb832](https://www.github.com/googleapis/python-bigquery/commit/5bbb832a83ebb66db4b5ee740cdfc53f4df8430b))
28+
29+
30+
### Documentation
31+
32+
* add code samples for Jupyter/IPython magics ([#1013](https://www.github.com/googleapis/python-bigquery/issues/1013)) ([61141ee](https://www.github.com/googleapis/python-bigquery/commit/61141ee0634024ad261d1595c95cd14a896fb87e))
33+
* **samples:** add create external table with hive partitioning ([#1033](https://www.github.com/googleapis/python-bigquery/issues/1033)) ([d64f5b6](https://www.github.com/googleapis/python-bigquery/commit/d64f5b682854a2293244426316890df4ab1e079e))
34+
35+
## [2.29.0](https://www.github.com/googleapis/python-bigquery/compare/v2.28.1...v2.29.0) (2021-10-27)
36+
37+
38+
### Features
39+
40+
* add `QueryJob.schema` property for dry run queries ([#1014](https://www.github.com/googleapis/python-bigquery/issues/1014)) ([2937fa1](https://www.github.com/googleapis/python-bigquery/commit/2937fa1386898766c561579fd39d42958182d260))
41+
* add session and connection properties to QueryJobConfig ([#1024](https://www.github.com/googleapis/python-bigquery/issues/1024)) ([e4c94f4](https://www.github.com/googleapis/python-bigquery/commit/e4c94f446c27eb474f30b033c1b62d11bd0acd98))
42+
* add support for INTERVAL data type to `list_rows` ([#840](https://www.github.com/googleapis/python-bigquery/issues/840)) ([e37380a](https://www.github.com/googleapis/python-bigquery/commit/e37380a959cbd5bb9cbbf6807f0a8ea147e0a713))
43+
* allow queryJob.result() to be called on a dryRun ([#1015](https://www.github.com/googleapis/python-bigquery/issues/1015)) ([685f06a](https://www.github.com/googleapis/python-bigquery/commit/685f06a5e7b5df17a53e9eb340ff04ecd1e51d1d))
44+
45+
46+
### Documentation
47+
48+
* document ScriptStatistics and other missing resource classes ([#1023](https://www.github.com/googleapis/python-bigquery/issues/1023)) ([6679109](https://www.github.com/googleapis/python-bigquery/commit/66791093c61f262ea063d2a7950fc643915ee693))
49+
* fix formatting of generated client docstrings ([#1009](https://www.github.com/googleapis/python-bigquery/issues/1009)) ([f7b0ee4](https://www.github.com/googleapis/python-bigquery/commit/f7b0ee45a664295ccc9f209eeeac122af8de3c80))
50+
51+
52+
### Dependencies
53+
54+
* allow pyarrow 6.x ([#1031](https://www.github.com/googleapis/python-bigquery/issues/1031)) ([1c2de74](https://www.github.com/googleapis/python-bigquery/commit/1c2de74a55046a343bcf9474f67100a82fb05401))
55+
856
### [2.28.1](https://www.github.com/googleapis/python-bigquery/compare/v2.28.0...v2.28.1) (2021-10-07)
957

1058

‎docs/conf.py‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -366,8 +366,9 @@
366366
"grpc": ("https://grpc.github.io/grpc/python/", None),
367367
"proto-plus": ("https://proto-plus-python.readthedocs.io/en/latest/", None),
368368
"protobuf": ("https://googleapis.dev/python/protobuf/latest/", None),
369-
"pandas": ("http://pandas.pydata.org/pandas-docs/stable/", None),
369+
"dateutil": ("https://dateutil.readthedocs.io/en/latest/", None),
370370
"geopandas": ("https://geopandas.org/", None),
371+
"pandas": ("https://pandas.pydata.org/pandas-docs/stable/", None),
371372
}
372373

373374

‎docs/magics.rst‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,34 @@
11
IPython Magics for BigQuery
22
===========================
33

4+
To use these magics, you must first register them. Run the ``%load_ext`` magic
5+
in a Jupyter notebook cell.
6+
7+
.. code::
8+
9+
%load_ext google.cloud.bigquery
10+
11+
This makes the ``%%bigquery`` magic available.
12+
13+
Code Samples
14+
------------
15+
16+
Running a query:
17+
18+
.. literalinclude:: ./samples/magics/query.py
19+
:dedent: 4
20+
:start-after: [START bigquery_jupyter_query]
21+
:end-before: [END bigquery_jupyter_query]
22+
23+
Running a parameterized query:
24+
25+
.. literalinclude:: ./samples/magics/query_params_scalars.py
26+
:dedent: 4
27+
:start-after: [START bigquery_jupyter_query_params_scalars]
28+
:end-before: [END bigquery_jupyter_query_params_scalars]
29+
30+
API Reference
31+
-------------
32+
433
.. automodule:: google.cloud.bigquery.magics.magics
534
:members:

‎google/cloud/bigquery/__init__.py‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -51,6 +51,7 @@
5151
from google.cloud.bigquery.external_config import ExternalSourceFormat
5252
from google.cloud.bigquery.format_options import AvroOptions
5353
from google.cloud.bigquery.format_options import ParquetOptions
54+
from google.cloud.bigquery.job.base import SessionInfo
5455
from google.cloud.bigquery.job import Compression
5556
from google.cloud.bigquery.job import CopyJob
5657
from google.cloud.bigquery.job import CopyJobConfig
@@ -76,6 +77,7 @@
7677
from google.cloud.bigquery.model import ModelReference
7778
from google.cloud.bigquery.query import ArrayQueryParameter
7879
from google.cloud.bigquery.query import ArrayQueryParameterType
80+
from google.cloud.bigquery.query import ConnectionProperty
7981
from google.cloud.bigquery.query import ScalarQueryParameter
8082
from google.cloud.bigquery.query import ScalarQueryParameterType
8183
from google.cloud.bigquery.query import SqlParameterScalarTypes
@@ -108,6 +110,7 @@
108110
"__version__",
109111
"Client",
110112
# Queries
113+
"ConnectionProperty",
111114
"QueryJob",
112115
"QueryJobConfig",
113116
"ArrayQueryParameter",
@@ -137,6 +140,7 @@
137140
"ExtractJobConfig",
138141
"LoadJob",
139142
"LoadJobConfig",
143+
"SessionInfo",
140144
"UnknownJob",
141145
# Models
142146
"Model",

‎google/cloud/bigquery/_helpers.py‎

Lines changed: 46 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,8 +19,9 @@
1919
import decimal
2020
import math
2121
import re
22-
from typing import Union
22+
from typing import Optional, Union
2323

24+
from dateutil import relativedelta
2425
from google.cloud._helpers import UTC
2526
from google.cloud._helpers import _date_from_iso8601_date
2627
from google.cloud._helpers import _datetime_from_microseconds
@@ -40,6 +41,14 @@
4041
re.VERBOSE,
4142
)
4243

44+
# BigQuery sends INTERVAL data in "canonical format"
45+
# https://cloud.google.com/bigquery/docs/reference/standard-sql/data-types#interval_type
46+
_INTERVAL_PATTERN = re.compile(
47+
r"(?P<calendar_sign>-?)(?P<years>\d+)-(?P<months>\d+) "
48+
r"(?P<days>-?\d+) "
49+
r"(?P<time_sign>-?)(?P<hours>\d+):(?P<minutes>\d+):(?P<seconds>\d+)\.?(?P<fraction>\d*)?$"
50+
)
51+
4352
_BQ_STORAGE_OPTIONAL_READ_SESSION_VERSION = packaging.version.Version("2.6.0")
4453

4554

@@ -116,6 +125,41 @@ def _int_from_json(value, field):
116125
return int(value)
117126

118127

128+
def _interval_from_json(
129+
value: Optional[str], field
130+
) -> Optional[relativedelta.relativedelta]:
131+
"""Coerce 'value' to an interval, if set or not nullable."""
132+
if not _not_null(value, field):
133+
return None
134+
if value is None:
135+
raise TypeError(f"got {value} for REQUIRED field: {repr(field)}")
136+
137+
parsed = _INTERVAL_PATTERN.match(value)
138+
if parsed is None:
139+
raise ValueError(f"got interval: '{value}' with unexpected format")
140+
141+
calendar_sign = -1 if parsed.group("calendar_sign") == "-" else 1
142+
years = calendar_sign * int(parsed.group("years"))
143+
months = calendar_sign * int(parsed.group("months"))
144+
days = int(parsed.group("days"))
145+
time_sign = -1 if parsed.group("time_sign") == "-" else 1
146+
hours = time_sign * int(parsed.group("hours"))
147+
minutes = time_sign * int(parsed.group("minutes"))
148+
seconds = time_sign * int(parsed.group("seconds"))
149+
fraction = parsed.group("fraction")
150+
microseconds = time_sign * int(fraction.ljust(6, "0")[:6]) if fraction else 0
151+
152+
return relativedelta.relativedelta(
153+
years=years,
154+
months=months,
155+
days=days,
156+
hours=hours,
157+
minutes=minutes,
158+
seconds=seconds,
159+
microseconds=microseconds,
160+
)
161+
162+
119163
def _float_from_json(value, field):
120164
"""Coerce 'value' to a float, if set or not nullable."""
121165
if _not_null(value, field):
@@ -252,6 +296,7 @@ def _record_from_json(value, field):
252296
_CELLDATA_FROM_JSON = {
253297
"INTEGER": _int_from_json,
254298
"INT64": _int_from_json,
299+
"INTERVAL": _interval_from_json,
255300
"FLOAT": _float_from_json,
256301
"FLOAT64": _float_from_json,
257302
"NUMERIC": _decimal_from_json,

0 commit comments

Comments
 (0)