Skip to content

Conversation

ravi-databricks
Copy link
Contributor

New Silver Quarantine Table Attributes Introduced in onboarding.json:

  • silver_catalog_quarantine
  • silver_database_quarantine
  • silver_quarantine_table
  • silver_quarantine_table_properties
  • silver_quarantine_cluster

Also Added:

To run tests:

python integration_tests/run_integration_tests.py --uc_catalog_name=<<catalog_name>> --source=cloudfiles

📊 Silver Onboarding DataflowSpec for customer and transaction feeds:

Image

🔁 Silver Layer DLT for customer and transaction tables:

Image Image
ravi-databricks and others added 6 commits July 23, 2025 11:59
1.Silver Quarantine table details to onboarding 2.Silver Quarantine table to dataflow_pipeline for regular flow 3.Unit test cases
1.Q table details for silver dataflow pipeline spec 2.Integration tests
Fixed unit tests for silver Q table
@ravi-databricks ravi-databricks added this to the v0.0.10 milestone Jul 24, 2025
@ravi-databricks ravi-databricks self-assigned this Jul 24, 2025
@ravi-databricks ravi-databricks added the enhancement New feature or request label Jul 24, 2025
@ravi-databricks ravi-databricks linked an issue Jul 24, 2025 that may be closed by this pull request
Copy link

@brij-raghuwanshi-db brij-raghuwanshi-db left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ravi-databricks ravi-databricks merged commit f3ac740 into feature/v0.0.10 Jul 31, 2025
1 of 2 checks passed
@ravi-databricks ravi-databricks mentioned this pull request Aug 16, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

2 participants