Skip to content

Conversation

@ravi-databricks
Copy link
Contributor

  • Added apply_changes_from_snapshot api support in bronze layer: PR
  • Added dlt append_flow api support for silver layer: PR
  • Added support for file metadata columns for autoloader: PR
  • Added support for Bring your own custom transformation: Issue
  • Added support to Unify PyPI releases with GitHub OIDC: PR
  • Added demo for append_flow and file_metadata options: PR
  • Added Demo for silver fanout architecture: PR
  • Added hugo-theme-relearn themee: PR
  • Added unit tests to showcase silver layer fanout examples: PR
  • Added liquid cluster support: PR
  • Added support for UC Volume + Serverless support for CLI, Integration tests and Demos: PR
  • Added Chaining bronze/silver pipelines into single DLT: PR
  • Fixed issue for No such file or directory: '/demo' :PR
  • Fixed issue DLT-META CLI onboard command issue for Azure: databricks.sdk.errors.platform.ResourceAlreadyExists :PR
  • Fixed issue Changed dbfs.create to mkdirs for CLI: PR
  • Fixed issue DLT-META CLI should use pypi lib instead of whl : PR
  • Fixed issue Onboarding with multiple partition columns errors out: PR
mithilmehta121287 and others added 30 commits August 12, 2024 14:46
Change Silver Configuration Parameter Co-Authored-By: Ravi Gawai <37003292+ravi-databricks@users.noreply.github.com>
…Config-Image-20240812 Update dltmeta_manual document
1.use uc volume paths to push config and resources 2.serverless for onboarding job and dlt
1.Updated all demo's to include unity catalog volumes path instead of dbfs 2.Updated all demo's to use serverless 3.Updated docs for demo options 4.Added demo workflow png files for DAIS and Techsummit demo
2.Added open_job_url function
1.CLI to include serverless for onboarding job 2.CLI to include uc volumes instead of dbfs for uc
1.Incorporated applyChangesFromSnapshot attribute into onboarding 2.Dataflowspec modified to incorporate applyChangesFromSnapshot 3.Unit tests
1.Removed cloud_provider variable from demo 2.Removed dbfs path from demos 3.Modified cloud_provider inside integration tests 4.Modified dbfs path inside inside integration tests
Merging integration tests with serverless and uc volume
…be made directly in editor and don't need to be modified in Databricks and downloaded as .dbc files
@brij-raghuwanshi-db
Copy link

Please check for hardcoding that can be changed or standardized.

@henry28zhang
Copy link

Looks good

Copy link

@ganeshchand ganeshchand left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ravi-databricks ravi-databricks removed the request for review from nsenno-dbr February 10, 2025 21:01
Copy link
Contributor

@pohlposition pohlposition left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@pohlposition pohlposition merged commit a234724 into main Feb 11, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment