Skip to content

Conversation

@ZAK1504
Copy link
Contributor

@ZAK1504 ZAK1504 commented Dec 12, 2025

Description

This PR implements a new adapter for Google BigQuery, enabling Intugle to connect to BigQuery datasets for profiling, analysis, and data product generation.

Changes Made

  • Created BigQuery adapter (src/intugle/adapters/types/bigquery/bigquery.py) with full implementation of all abstract methods from the Adapter base class
  • Added Pydantic models (models.py) for BigQuery connection configuration and data configuration
  • Registered adapter in factory with conditional loading based on dependency availability
  • Added optional dependency google-cloud-bigquery>=3.11.0 to pyproject.toml
  • Created comprehensive unit tests with 20 test cases covering adapter contract, BigQuery-specific behavior, error handling, and model validation
  • Added documentation (README.md) with usage examples, configuration guide, and troubleshooting tips

Features Implemented

✅ Connect to BigQuery using service accounts or Application Default Credentials
✅ Profile BigQuery tables and columns with detailed statistics
✅ Execute Standard SQL queries
✅ Convert query results to pandas DataFrames
✅ Create views and tables from queries (materialization)
✅ Analyze relationships between tables (intersection analysis)
✅ Support for composite keys
✅ Graceful handling of missing dependencies

Technical Implementation

  • Follows established adapter pattern (PostgresAdapter as reference)
  • Uses google-cloud-bigquery Python client library
  • Implements singleton pattern for adapter instance
  • Supports Standard SQL dialect
  • Compatible with Intugle's semantic search and data product generation
  • Handles Google Cloud authentication (service accounts + ADC)

Testing

  • Unit Tests: 20 comprehensive test cases
  • Tests Status: Skip when BigQuery dependencies not installed (expected behavior)
  • Coverage: Adapter contract, BigQuery-specific methods, error handling, model validation

License Compliance

google-cloud-bigquery uses Apache License 2.0 (fully compatible with project's Apache 2.0 license)

Breaking Changes

None - this is a new feature addition

Related Issue

Closes #117 - [HELP WANTED] Implement a new adapter for Google BigQuery

@shiva-intugle shiva-intugle merged commit 707d742 into Intugle:main Dec 18, 2025
1 check passed
@raphael-intugle
Copy link
Collaborator

Superb work @ZAK1504 !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

3 participants