fix(databricks-driver): exclude partition information metadata #10262
+7 −2
Add this suggestion to a batch that can be applied as a single commit. This suggestion is invalid because no changes were made to the code. Suggestions cannot be applied while the pull request is closed. Suggestions cannot be applied while viewing a subset of changes. Only one suggestion per line can be applied in a batch. Add this suggestion to a batch that can be applied as a single commit. Applying suggestions on deleted lines is not supported. You must change the existing code in this line in order to create a valid suggestion. Outdated suggestions cannot be applied. This suggestion has been applied or marked resolved. Suggestions cannot be applied from pending reviews. Suggestions cannot be applied on multi-line comments. Suggestions cannot be applied while the pull request is queued to merge. Suggestion cannot be applied right now. Please check back later.
Check List
Issue Reference this PR resolves
N/A
Description of Changes Made
Databricks
DESCRIBEoutput for partitioned tables includes metadata rows where the column name is# Partition Informationand thedata_typefield is empty (docs.databricks). These rows were previously parsed as columns.This change stops column parsing for the partition metadata section, ensuring that only actual table columns are returned. The check requires both the column name to match
# Partition Informationand*thedata_typeto be empty, as a real table column could legitimately have this name but would always include a non-emptydata_type.Issue Explanation
Selecting YAML for

Generate Data Modelcauses a playground error because the generated YAML becomes invalid when partition metadata rows are included as columns.Playground error.

Selecting

JavaScriptdoes not trigger a playground error, but the generated schema still includescol_nameand# Partition Information, which should not be treated as table columns.