Skip to content

Conversation

@sjedhoff
Copy link

Previously, only calibration_ecdf supported custom test quantities. This PR adds that capability to additional diagnostic metrics and plotting functions.

Updated Metrics:

  • calibration_error
  • calibration_log_gamma
  • posterior_contraction
  • rott_mean_squared_error

Updated Plots:

  • calibration_histogram
  • coverage
  • recovery
  • z_score_contraction

No breaking changes; existing behavior is preserved when no custom test quantities are provided.

@paul-buerkner paul-buerkner self-requested a review October 15, 2025 12:43
@codecov
Copy link

codecov bot commented Oct 15, 2025

Codecov Report

❌ Patch coverage is 91.66667% with 5 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
bayesflow/diagnostics/plots/recovery.py 28.57% 5 Missing ⚠️
Files with missing lines Coverage Δ
bayesflow/diagnostics/metrics/calibration_error.py 100.00% <100.00%> (ø)
...sflow/diagnostics/metrics/calibration_log_gamma.py 100.00% <100.00%> (ø)
...sflow/diagnostics/metrics/posterior_contraction.py 100.00% <100.00%> (ø)
...low/diagnostics/metrics/root_mean_squared_error.py 60.00% <100.00%> (+7.05%) ⬆️
...yesflow/diagnostics/plots/calibration_histogram.py 92.50% <100.00%> (+1.59%) ⬆️
bayesflow/diagnostics/plots/coverage.py 97.61% <100.00%> (+0.11%) ⬆️
bayesflow/diagnostics/plots/z_score_contraction.py 96.66% <100.00%> (+1.01%) ⬆️
bayesflow/diagnostics/plots/recovery.py 82.05% <28.57%> (-11.70%) ⬇️

... and 2 files with indirect coverage changes

Copy link
Contributor

@paul-buerkner paul-buerkner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Thank you!

@paul-buerkner paul-buerkner merged commit d13da56 into bayesflow-org:dev Oct 16, 2025
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

2 participants