Normalization: check for ref type existence #22161
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Contributor
Affected Connector ReportNOTE
|
| Connector | Version | Changelog | Publish |
|---|
- See "Actionable Items" below for how to resolve warnings and errors.
❌ Destinations (16)
| Connector | Version | Changelog | Publish |
|---|---|---|---|
destination-bigquery | 1.2.13 | ✅ | ✅ |
destination-bigquery-denormalized | 1.2.12 | ✅ | ❌ (diff seed version) |
destination-clickhouse | 0.2.2 | ❌ (changelog missing) | ✅ |
destination-clickhouse-strict-encrypt | 0.2.2 | 🔵 (ignored) | 🔵 (ignored) |
destination-jdbc | 0.3.14 | 🔵 (ignored) | 🔵 (ignored) |
destination-mssql | 0.1.22 | ✅ | ✅ |
destination-mssql-strict-encrypt | 0.1.22 | 🔵 (ignored) | 🔵 (ignored) |
destination-mysql | 0.1.20 | ✅ | ✅ |
destination-mysql-strict-encrypt | ❌ 0.1.21(mismatch: 0.1.20) | 🔵 (ignored) | 🔵 (ignored) |
destination-oracle | 0.1.19 | ✅ | ✅ |
destination-oracle-strict-encrypt | 0.1.19 | 🔵 (ignored) | 🔵 (ignored) |
destination-postgres | 0.3.26 | ✅ | ✅ |
destination-postgres-strict-encrypt | 0.3.26 | 🔵 (ignored) | 🔵 (ignored) |
destination-redshift | 0.3.56 | ✅ | ✅ |
destination-snowflake | 0.4.47 | ✅ | ✅ |
destination-tidb | 0.1.0 | ✅ | ✅ |
- See "Actionable Items" below for how to resolve warnings and errors.
👀 Other Modules (1)
- base-normalization
Actionable Items
(click to expand)
| Category | Status | Actionable Item |
|---|---|---|
| Version | ❌ mismatch | The version of the connector is different from its normal variant. Please bump the version of the connector. |
| ⚠ doc not found | The connector does not seem to have a documentation file. This can be normal (e.g. basic connector like source-jdbc is not published or documented). Please double-check to make sure that it is not a bug. | |
| Changelog | ⚠ doc not found | The connector does not seem to have a documentation file. This can be normal (e.g. basic connector like source-jdbc is not published or documented). Please double-check to make sure that it is not a bug. |
| ❌ changelog missing | There is no chnagelog for the current version of the connector. If you are the author of the current version, please add a changelog. | |
| Publish | ⚠ not in seed | The connector is not in the seed file (e.g. source_definitions.yaml), so its publication status cannot be checked. This can be normal (e.g. some connectors are cloud-specific, and only listed in the cloud seed file). Please double-check to make sure that it is not a bug. |
| ❌ diff seed version | The connector exists in the seed file, but the latest version is not listed there. This usually means that the latest version is not published. Please use the /publish command to publish the latest version. |
gosusnp approved these changes Jan 31, 2023
octavia-squidington-iv added the area/documentation 
Contributor Author
| skipping tests for urgent publish |
Contributor Author
| /publish connector=bases/base-normalization run-tests=false
if you have connectors that successfully published but failed definition generation, follow step 4 here |
gosusnp approved these changes Jan 31, 2023
Contributor
Airbyte Code Coverage
|
jbfbell approved these changes Jan 31, 2023
| ) | ||
| | ||
| | ||
| def is_combining_node(properties: dict) -> Set[str]: |
Contributor
There was a problem hiding this comment.
Some really unfair comments here so sorry in advance (none are blocking):
is_combining_nodereads like it should be a boolean, but it returns a set?- As a new observer, I understand what its doing (checks a condition and returns an empty or populated set) but i have no idea why its doing that or what a "combining node" is, some docs on the method could be useful here
- is there a test for this?
- I don't feel too strongly about this but the return values could be static but this is a super nitpick
edgao added a commit that referenced this pull request Feb 6, 2023
* Revert "Normalization: handle non-object top-level schemas; treat binary data as string (#22165)" This reverts commit 8276d03. * Revert "Normalization: check for ref type existence (#22161)" This reverts commit dbe56d6. * Revert "🎉Updated normalization to handle new datatypes (#19721)" This reverts commit c1d7736. * revert dest definitions * also dockerfile * re-add to changelog * add comment in dockerfile
This was referenced Feb 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Add this suggestion to a batch that can be applied as a single commit. This suggestion is invalid because no changes were made to the code. Suggestions cannot be applied while the pull request is closed. Suggestions cannot be applied while viewing a subset of changes. Only one suggestion per line can be applied in a batch. Add this suggestion to a batch that can be applied as a single commit. Applying suggestions on deleted lines is not supported. You must change the existing code in this line in order to create a valid suggestion. Outdated suggestions cannot be applied. This suggestion has been applied or marked resolved. Suggestions cannot be applied from pending reviews. Suggestions cannot be applied on multi-line comments. Suggestions cannot be applied while the pull request is queued to merge. Suggestion cannot be applied right now. Please check back later.
fix this error