Billing integration dedup credits & stalling fixes #12796
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
…duplicate credit count issue
efd6 reviewed Feb 19, 2025
packages/ess_billing/data_stream/credits/agent/stream/cel.yml.hbs Outdated Show resolved Hide resolved
💚 Build Succeeded
History
|
|
ryanburnhamelastic approved these changes Feb 20, 2025
| Package ess_billing - 1.1.0 containing this change is available at https://epr.elastic.co/package/ess_billing/1.1.0/ |
andrewkroh added Integration:ess_billing flexitrev pushed a commit that referenced this pull request Mar 20, 2025
* Changed credits collection mechanism, fixed stalling issue and fixed duplicate credit count issue * Escaped double quotes in dashboard vega * Correct number in changelog entry * Credit fingerprinting * Return a document even when no cost is given * Removed interval configuration * Refactored billing CEL code * Fixed timestamp parsing * Fixed parameter handling * Added time filter in credit view * Fixed total balance in credits dashboard * Use signal event dropped in the ingest pipeline * Wrap credits CEL code in state.with * Ingest pipeline typo
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit. This suggestion is invalid because no changes were made to the code. Suggestions cannot be applied while the pull request is closed. Suggestions cannot be applied while viewing a subset of changes. Only one suggestion per line can be applied in a batch. Add this suggestion to a batch that can be applied as a single commit. Applying suggestions on deleted lines is not supported. You must change the existing code in this line in order to create a valid suggestion. Outdated suggestions cannot be applied. This suggestion has been applied or marked resolved. Suggestions cannot be applied from pending reviews. Suggestions cannot be applied on multi-line comments. Suggestions cannot be applied while the pull request is queued to merge. Suggestion cannot be applied right now. Please check back later.
Proposed commit message
This fixes multiple bugs, while slightly changing the collection behavior for credits data.
The wrong credit computation was caused by the lack of deduplication. This is fixed by ensuring we only look at the last 24h, and controlling the IDs within 24h (edge case with multiple collecting agents, or agent full reinstall).
The stalling collection was caused by a missing
api_keycarry-over when an error is faced. As a nice benefit, i've also ensured that an error while back-feeding doesn't completely stops the collection, ie the agent will attempt to move forward until "live" in spite of the error(s).TODOs
organization_id/daily timestamp/ecu_quantity/start(assuming we never have 2 orders on the same date with the exact amount of ECUs)want_moreis true (reference)Stalling collection fix
The integration will now return "empty" documents if no cost is returned for the point of collection. This ensures the
want_morebehavior is enforced.