Skip to content

Commit 0bd9d52

Browse files
fix: remove local copy of beam I/O now that it is merged to upstream (#873)
* fix: remove local copy of beam I/O now that it is merged to upstream * fix: add README back and update * chore: cleanup after sync * chore: cleanup after sync
1 parent 61d17ef commit 0bd9d52

File tree

58 files changed

+12
-4667
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

58 files changed

+12
-4667
lines changed

README.md

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -372,9 +372,6 @@ Samples are in the [`samples/`](https://github.com/googleapis/java-pubsublite/tr
372372
| Subscriber Example | [source code](https://github.com/googleapis/java-pubsublite/blob/master/samples/snippets/src/main/java/pubsublite/SubscriberExample.java) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/java-pubsublite&page=editor&open_in_editor=samples/snippets/src/main/java/pubsublite/SubscriberExample.java) |
373373
| Update Subscription Example | [source code](https://github.com/googleapis/java-pubsublite/blob/master/samples/snippets/src/main/java/pubsublite/UpdateSubscriptionExample.java) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/java-pubsublite&page=editor&open_in_editor=samples/snippets/src/main/java/pubsublite/UpdateSubscriptionExample.java) |
374374
| Update Topic Example | [source code](https://github.com/googleapis/java-pubsublite/blob/master/samples/snippets/src/main/java/pubsublite/UpdateTopicExample.java) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/java-pubsublite&page=editor&open_in_editor=samples/snippets/src/main/java/pubsublite/UpdateTopicExample.java) |
375-
| Read Messages Example | [source code](https://github.com/googleapis/java-pubsublite/blob/master/samples/snippets/src/main/java/pubsublite/beam/ReadMessagesExample.java) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/java-pubsublite&page=editor&open_in_editor=samples/snippets/src/main/java/pubsublite/beam/ReadMessagesExample.java) |
376-
| Write Messages Example | [source code](https://github.com/googleapis/java-pubsublite/blob/master/samples/snippets/src/main/java/pubsublite/beam/WriteMessagesExample.java) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/java-pubsublite&page=editor&open_in_editor=samples/snippets/src/main/java/pubsublite/beam/WriteMessagesExample.java) |
377-
378375

379376

380377
## Troubleshooting

pom.xml

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -152,7 +152,6 @@
152152
<module>google-cloud-pubsublite</module>
153153
<module>grpc-google-cloud-pubsublite-v1</module>
154154
<module>proto-google-cloud-pubsublite-v1</module>
155-
<module>pubsublite-beam-io</module>
156155
<module>google-cloud-pubsublite-bom</module>
157156
</modules>
158157

pubsublite-beam-io/README.md

Lines changed: 12 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -1,28 +1,27 @@
11
# Instructions for PubsubLiteIO usage.
22

3-
NOTE: Drain is not currently supported on `PubsubLiteIO`. Once Beam 2.29.0 is released, this will be supported
4-
only when `--experiments=use_runner_v2` is enabled. Please contact dataflow customer support to request adding projects
5-
into the allowlist and launch the pipeline with `--experiments=use_runner_v2`.
3+
The minimum version you should use for Pub/Sub Lite IO with beam is 2.34.0.
64

7-
1. Add the following to your `pom.xml` file to download the Pub/Sub Lite I/O.
5+
1. Add the following to your `pom.xml` file to download the Pub/Sub Lite I/O
6+
included with Beam.
87
```xml
98
<dependency>
10-
<groupId>com.google.cloud</groupId>
11-
<artifactId>pubsublite-beam-io</artifactId>
12-
<version>0.15.0</version>
9+
<groupId>org.apache.beam</groupId>
10+
<artifactId>beam-sdks-java-io-google-cloud-platform</artifactId>
11+
<version>2.34.0</version>
1312
</dependency>
1413
```
1514
1. Create a topic using [`gcloud pubsub lite-topics create`](https://cloud.google.com/sdk/gcloud/reference/pubsub/lite-topics/create)
1615
1. Write some messages using:
1716

1817
```java
19-
import com.google.cloud.pubsublite.beam.PubsubLiteIO;
20-
import com.google.cloud.pubsublite.beam.PublisherOptions;
2118
import com.google.cloud.pubsublite.proto.PubSubMessage;
2219
import com.google.cloud.pubsublite.CloudZone;
2320
import com.google.cloud.pubsublite.ProjectIdOrNumber;
2421
import com.google.cloud.pubsublite.TopicName;
2522
import com.google.cloud.pubsublite.TopicPath;
23+
import org.apache.beam.sdk.io.gcp.pubsublite.PubsubLiteIO;
24+
import org.apache.beam.sdk.io.gcp.pubsublite.PublisherOptions;
2625

2726
...
2827

@@ -45,13 +44,13 @@ into the allowlist and launch the pipeline with `--experiments=use_runner_v2`.
4544
1. Read some messages using:
4645

4746
```java
48-
import com.google.cloud.pubsublite.beam.PubsubLiteIO;
49-
import com.google.cloud.pubsublite.beam.SubscriberOptions;
5047
import com.google.cloud.pubsublite.proto.SequencedMessage;
5148
import com.google.cloud.pubsublite.CloudZone;
5249
import com.google.cloud.pubsublite.ProjectIdOrNumber;
5350
import com.google.cloud.pubsublite.SubscriptionName;
5451
import com.google.cloud.pubsublite.SubscriptionPath;
52+
import org.apache.beam.sdk.io.gcp.pubsublite.PubsubLiteIO;
53+
import org.apache.beam.sdk.io.gcp.pubsublite.SubscriberOptions;
5554

5655
...
5756

@@ -72,16 +71,5 @@ into the allowlist and launch the pipeline with `--experiments=use_runner_v2`.
7271
```
7372

7473
### Known issues and workarounds
75-
1. If you encounter `IllegalAccessError` as shown below, place the Pub/Sub Lite
76-
I/O connector as the first dependency before all other Beam dependencies in
77-
your `pom.xml`.
78-
79-
>`java.lang.IllegalAccessError: failed to access class com.google.cloud.
80-
pubsublite.internal.wire.RoutingMetadata from class com.google.cloud.
81-
pubsublite.beam.SubscriberOptions (com.google.cloud.pubsublite.internal.
82-
wire.RoutingMetadata and com.google.cloud.pubsublite.beam.SubscriberOptions
83-
are in unnamed module of loader java.net.URLClassLoader @ff2266c)`
84-
85-
1. Aggregate transforms after fixed windowing will not emit elements in the
86-
DirectRunner for Pub/Sub Lite I/O unless you explicitly fire after watermark
87-
triggers, or use processing-time triggers instead.
74+
1. Drain does not work on the default dataflow runner. You must set the option
75+
`--experiments=use_runner_v2` for draining to function correctly.

pubsublite-beam-io/clirr-ignored-differences.xml

Lines changed: 0 additions & 19 deletions
This file was deleted.

pubsublite-beam-io/pom.xml

Lines changed: 0 additions & 174 deletions
This file was deleted.

pubsublite-beam-io/src/main/java/com/google/cloud/pubsublite/beam/AddUuidsTransform.java

Lines changed: 0 additions & 48 deletions
This file was deleted.

pubsublite-beam-io/src/main/java/com/google/cloud/pubsublite/beam/CloudPubsubMessages.java

Lines changed: 0 additions & 60 deletions
This file was deleted.

0 commit comments

Comments
 (0)