- Notifications
You must be signed in to change notification settings - Fork 933
[DTX-872] Implement Async Schema Registry client #1965
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
| 🎉 All Contributor License Agreements have been signed. Ready to merge. |
30156d9 to 920ddb6 Compare cd95e33 to bd445fd Compare 036451a to 1fe184e Compare …support src/confluent_kafka/schema_registry/avro.py (change applied to src/confluent_kafka/schema_registry/_sync/avro.py)
1fe184e to 0255917 Compare 0255917 to 3181809 Compare 3181809 to 082df8f Compare There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Async implementation of the Kafka consumer -- used only in tests and not surfaced as a public API.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Async implementation of the Kafka producer -- used only in tests and not surfaced as a public API.
| per-file-ignores = | ||
| ./src/confluent_kafka/schema_registry/_sync/avro.py: E303 | ||
| ./src/confluent_kafka/schema_registry/_sync/json_schema.py: E303 | ||
| ./src/confluent_kafka/schema_registry/_sync/protobuf.py: E303 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
With @asyncinit in async files replaced with an empty string in the sync files, this leaves three empty lines separating the classes which is a flake8 violation.
c5c548e to dac2bef Compare This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
rayokota left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @rohitsanj , LGTM
rayokota left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
| poll_timeout = None if poll_timeout == -1 else poll_timeout | ||
| async with timeout(poll_timeout): | ||
| while True: | ||
| # Zero timeout here is what makes it non-blocking |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Remove this because it's wrong -- poll(0) could block.
Edit: Not pressing, can be handled later.
What
_asyncmodule.await func(x, y, z)->func(x, y, z),AsyncXYZ->XYZand so on). Going forward, all diffs must be solely made to the_asyncmodule, the sync clients will be derived from the async clients.#1971 must be merged before this one.
unasync.pyscriptScenario 1: When changes are made to the _async directory but not to its _sync counterpart:
Scenario 2: When changes across _async and _sync are consistent.
Checklist
References
JIRA:
Test & Review
Open questions / Follow-ups