Skip to content

Commit 3ec294f

Browse files
fix: remove teh 30 minutes wait trying to refresh schema. Customers s… (#2088)
* fix: remove teh 30 minutes wait trying to refresh schema. Customers should be ready to handle InvalidArgumentErrors, and removing the wait and retry can reduce their chance to get stuck * . * . * . * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
1 parent f6651dc commit 3ec294f

File tree

3 files changed

+6
-25
lines changed

3 files changed

+6
-25
lines changed

README.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -57,13 +57,13 @@ implementation 'com.google.cloud:google-cloud-bigquerystorage'
5757
If you are using Gradle without BOM, add this to your dependencies:
5858

5959
```Groovy
60-
implementation 'com.google.cloud:google-cloud-bigquerystorage:2.35.0'
60+
implementation 'com.google.cloud:google-cloud-bigquerystorage:2.36.0'
6161
```
6262

6363
If you are using SBT, add this to your dependencies:
6464

6565
```Scala
66-
libraryDependencies += "com.google.cloud" % "google-cloud-bigquerystorage" % "2.35.0"
66+
libraryDependencies += "com.google.cloud" % "google-cloud-bigquerystorage" % "2.36.0"
6767
```
6868
<!-- {x-version-update-end} -->
6969

@@ -220,7 +220,7 @@ Java is a registered trademark of Oracle and/or its affiliates.
220220
[kokoro-badge-link-5]: http://storage.googleapis.com/cloud-devrel-public/java/badges/java-bigquerystorage/java11.html
221221
[stability-image]: https://img.shields.io/badge/stability-stable-green
222222
[maven-version-image]: https://img.shields.io/maven-central/v/com.google.cloud/google-cloud-bigquerystorage.svg
223-
[maven-version-link]: https://central.sonatype.com/artifact/com.google.cloud/google-cloud-bigquerystorage/2.35.0
223+
[maven-version-link]: https://central.sonatype.com/artifact/com.google.cloud/google-cloud-bigquerystorage/2.36.0
224224
[authentication]: https://github.com/googleapis/google-cloud-java#authentication
225225
[auth-scopes]: https://developers.google.com/identity/protocols/oauth2/scopes
226226
[predefined-iam-roles]: https://cloud.google.com/iam/docs/understanding-roles#predefined_roles

google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1/SchemaAwareStreamWriter.java

Lines changed: 2 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -44,7 +44,6 @@
4444
*/
4545
public class SchemaAwareStreamWriter<T> implements AutoCloseable {
4646
private static final Logger LOG = Logger.getLogger(SchemaAwareStreamWriter.class.getName());
47-
private static final long UPDATE_SCHEMA_RETRY_INTERVAL_MILLIS = 30100L;
4847
private final BigQueryWriteClient client;
4948
private final String streamName;
5049
private final StreamWriter.Builder streamWriterBuilder;
@@ -126,9 +125,6 @@ private Message buildMessage(T item)
126125
return this.toProtoConverter.convertToProtoMessage(
127126
this.descriptor, this.tableSchema, item, ignoreUnknownFields);
128127
} catch (Exceptions.DataHasUnknownFieldException ex) {
129-
// Backend cache for GetWriteStream schema staleness can be 30 seconds, wait a bit before
130-
// trying to get the table schema to increase the chance of succeed. This is to avoid
131-
// client's invalid datfa caused storm of GetWriteStream.
132128
LOG.warning(
133129
"Saw unknown field "
134130
+ ex.getFieldName()
@@ -141,21 +137,8 @@ private Message buildMessage(T item)
141137
.build();
142138
WriteStream writeStream = client.getWriteStream(writeStreamRequest);
143139
refreshWriter(writeStream.getTableSchema());
144-
try {
145-
return this.toProtoConverter.convertToProtoMessage(
146-
this.descriptor, this.tableSchema, item, ignoreUnknownFields);
147-
} catch (Exceptions.DataHasUnknownFieldException exex) {
148-
LOG.warning(
149-
"First attempt failed, waiting for 30 seconds to retry, stream: " + this.streamName);
150-
Thread.sleep(UPDATE_SCHEMA_RETRY_INTERVAL_MILLIS);
151-
writeStream = client.getWriteStream(writeStreamRequest);
152-
// TODO(yiru): We should let TableSchema return a timestamp so that we can simply
153-
// compare the timestamp to see if the table schema is the same. If it is the
154-
// same, we don't need to go refresh the writer again.
155-
refreshWriter(writeStream.getTableSchema());
156-
return this.toProtoConverter.convertToProtoMessage(
157-
this.descriptor, this.tableSchema, item, ignoreUnknownFields);
158-
}
140+
return this.toProtoConverter.convertToProtoMessage(
141+
this.descriptor, this.tableSchema, item, ignoreUnknownFields);
159142
}
160143
}
161144
/**

google-cloud-bigquerystorage/src/test/java/com/google/cloud/bigquery/storage/v1/JsonStreamWriterTest.java

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -799,9 +799,7 @@ public void testWithoutIgnoreUnknownFieldsUpdateSecondSuccess() throws Exception
799799
.setType(TableFieldSchema.Type.STRING)
800800
.setMode(Mode.NULLABLE))
801801
.build();
802-
// GetWriteStream is called twice and got the updated schema
803-
testBigQueryWrite.addResponse(
804-
WriteStream.newBuilder().setName(TEST_STREAM).setTableSchema(tableSchema).build());
802+
// GetWriteStream is called once and got the updated schema
805803
testBigQueryWrite.addResponse(
806804
WriteStream.newBuilder().setName(TEST_STREAM).setTableSchema(updatedSchema).build());
807805
testBigQueryWrite.addResponse(

0 commit comments

Comments
 (0)