Skip to content

Conversation

tobyhede
Copy link
Contributor

Summary

This PR introduces pgTAP as a robust testing framework for EQL, providing industry-standard PostgreSQL testing capabilities alongside the existing custom assertion helpers.

What's Included

Task 1: Install and Configure pgTAP ✅

  • Docker Integration: Created Dockerfile.pgtap that extends official PostgreSQL images (14-17) and installs pgTAP 1.3.5
  • Docker Compose: Updated docker-compose.yml to build custom images with pgTAP for all PostgreSQL versions
  • Installation Script: Added tests/install_pgtap.sql to install and verify the pgTAP extension
  • Test Runner: Created tasks/test-pgtap.sh - a mise task to run pgTAP tests with PostgreSQL version selection

Task 2: Structure Testing Suite ✅

Created comprehensive structure tests in tests/pgtap/structure/:

  • schema_test.sql (10 tests): Verifies eql_v2 schema, eql_v2_encrypted type, and eql_v2_configuration table structure
  • types_test.sql (7 tests): Verifies all index term types exist (blake3, hmac_256, bloom_filter, ORE variants)
  • functions_test.sql (9 tests): Verifies key functions exist with correct signatures (comparison, config management)
  • operators_test.sql (1 test): Verifies operators exist for eql_v2_encrypted type

All structure tests passing ✓

Task 3: Functionality Testing Suite (Partial) ✅

  • equality_test.sql (13 tests): Converted existing equality operator tests to pgTAP format
    • Tests eql_v2_encrypted = eql_v2_encrypted with HMAC and Blake3 indexes
    • Tests eql_v2.eq() function
    • Tests eql_v2_encrypted = jsonb and jsonb = eql_v2_encrypted operations
    • Includes both positive (matching) and negative (no match) test cases

Testing

All tests pass successfully:

mise run test # Existing tests still pass

pgTAP tests can be run with:

mise run test-pgtap

Benefits

  • Industry Standard: pgTAP follows the TAP (Test Anything Protocol) standard used across the PostgreSQL ecosystem
  • Rich Assertions: Provides specialized assertions for schema testing, function signatures, and more
  • Better Organization: Clear separation between structure tests and functionality tests
  • CI Integration: TAP output format enables easy integration with CI/CD pipelines
  • Maintainability: Well-documented testing framework makes it easier for contributors

Implementation Plan

This PR implements Tasks 1-3 from docs/pgtap-implementation-plan.md. Remaining tasks:

  • Task 3.2-3.5: Additional functionality tests (comparison, JSONB operators, config, index terms)
  • Task 4: Data seeding patterns documentation
  • Task 5: CI pipeline integration
  • Task 6: Testing documentation and migration guides
  • Task 7: Advanced testing enhancements

Related

Based on the implementation plan in docs/pgtap-implementation-plan.md

Implement pgTAP test framework to provide industry-standard PostgreSQL testing capabilities alongside existing custom assertion helpers. Changes: - Add pgTAP to Docker containers (PostgreSQL 14-17) - Create pgTAP test runner script (tasks/test-pgtap.sh) - Add structure tests: schema, types, functions, operators - Add functionality tests: equality operators - Update docker-compose.yml to build custom images with pgTAP Structure tests verify EQL schema elements exist with correct signatures. Functionality tests validate encrypted data equality operations using pgTAP assertions. This implements Tasks 1-3 from docs/pgtap-implementation-plan.md.
Complete Task 3 from pgTAP implementation plan by adding functionality tests for: - Comparison operators: <, <=, >, >=, <> with ORE indexes (12 tests) - JSONB operators: ->, ->>, @>, <@ for encrypted data (13 tests) - Configuration management: add/remove/modify search config (12 tests) - Index terms: blake3, hmac_256, and ORE comparison functions (27 tests) Total: 64 additional tests covering core EQL functionality These tests validate: - Encrypted data comparison operations work correctly - JSONB extraction and containment operators function as expected - Configuration management maintains state correctly - Index term comparison functions return proper ordering results All tests use pgTAP assertions (ok, is, cmp_ok, lives_ok, isnt_empty) and follow the established pattern of setup/test/cleanup.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

1 participant