Skip to content

test: fix dbcrash and fee_estimation extended tests #1298

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
delta1 wants to merge 3 commits into
base: master
Choose a base branch
from
Open

test: fix dbcrash and fee_estimation extended tests #1298

delta1 wants to merge 3 commits into from
+34 −13

Conversation

@delta1
Copy link
Member

@delta1 delta1 commented Dec 1, 2023

It seems these tests have never been run for Elements, since the extended tests are not run in the CI process.

This PR fixes both of them, however there may be a further issue with feature_dbcrash.py running for an extremely long time which we are still in the process of testing.

#1297
#1296

cc @whitslack
@delta1
Copy link
Member Author

delta1 commented Dec 8, 2023 •
edited
Loading

Force pushed to reduce the number of iterations in the dbcrash test, such that it still passes but in a "reasonable" amount of time (about 6 hours in my testing).

I have tested these 2 tests manually and they are now passing.

2023-12-08T19:50:34.715000Z TestFramework (INFO): Tests successful real 350m18.127s user 348m12.078s sys 1m56.746s
@delta1 delta1 mentioned this pull request Dec 8, 2023
Open
@jamesdorfman
Copy link
Collaborator

jamesdorfman commented Apr 4, 2024 •
edited
Loading

@delta1 am I running the correct test?
It's not passing for me -- I see the following:

$ ./test/functional/feature_dbcrash.py 2024-04-03T20:32:29.724000Z TestFramework (INFO): Initializing test directory /tmp/bitcoin_func_test_ikctzo2y 2024-04-03T21:35:24.776000Z TestFramework (INFO): Prepped 5001 utxo entries 2024-04-03T21:36:00.556000Z TestFramework (INFO): Iteration 0, generating 2500 transactions [0, 0, 0] 2024-04-03T23:10:26.156000Z TestFramework (INFO): Iteration 1, generating 2500 transactions [0, 0, 0] 2024-04-04T00:45:22.052000Z TestFramework (INFO): Iteration 2, generating 2500 transactions [1, 0, 0] 2024-04-04T02:20:44.177000Z TestFramework (INFO): Verifying utxo hash matches for all nodes 2024-04-04T02:20:45.540000Z TestFramework (INFO): Restarted nodes: [1, 0, 0]; crashes on restart: 0 2024-04-04T02:20:45.540000Z TestFramework (ERROR): Assertion failed Traceback (most recent call last): File "/home/james/elements/test/functional/test_framework/test_framework.py", line 132, in main self.run_test() File "/home/james/elements/./test/functional/feature_dbcrash.py", line 285, in run_test assert self.crashed_on_restart > 0 ^^^^^^^^^^^^^^^^^^^^^^^^^^^ AssertionError 2024-04-04T02:20:45.600000Z TestFramework (INFO): Stopping nodes 2024-04-04T02:20:45.856000Z TestFramework (WARNING): Not cleaning up dir /tmp/bitcoin_func_test_ikctzo2y 2024-04-04T02:20:45.856000Z TestFramework (ERROR): Test failed. Test logging available at /tmp/bitcoin_func_test_ikctzo2y/test_framework.log 2024-04-04T02:20:45.856000Z TestFramework (ERROR): 2024-04-04T02:20:45.857000Z TestFramework (ERROR): Hint: Call /home/james/byron-elements-clone/elements/test/functional/combine_logs.py '/tmp/bitcoin_func_test_ikctzo2y' to consolidate all logs 2024-04-04T02:20:45.857000Z TestFramework (ERROR): 2024-04-04T02:20:45.857000Z TestFramework (ERROR): If this failure happened unexpectedly or intermittently, please file a bug and provide a link or upload of the combined log. 2024-04-04T02:20:45.857000Z TestFramework (ERROR): https://github.com/ElementsProject/elements/issues 2024-04-04T02:20:45.857000Z TestFramework (ERROR): 
$ git log commit 5af2426c14fe62f7b60b84586221cc531251eac8 (HEAD -> issues-1297, origin/issues-1297) Author: Byron Hambly <bhambly@blockstream.com> Date: Fri Dec 1 09:19:34 2023 +0200 test: fix dbcrash and fee_estimation extended tests reduces the number of iterations for the dbcrash test so that it "only" takes 6 hours
@delta1
Copy link
Member Author

delta1 commented Apr 4, 2024 •
edited
Loading

@jamesdorfman yes you ran the right test. It seems I've reduced the values too much for it to fail (correctly) every time.

Bumped the number of iterations from 3 to 4, please test again. It will take closer to 8 hours now.
@delta1
Copy link
Member Author

delta1 commented Apr 4, 2024

latest run:

> test/functional/feature_dbcrash.py 2024-04-04T09:04:35.877000Z TestFramework (INFO): Initializing test directory /tmp/bitcoin_func_test_dybbv4yh 2024-04-04T09:48:20.887000Z TestFramework (INFO): Prepped 5001 utxo entries 2024-04-04T09:48:47.443000Z TestFramework (INFO): Iteration 0, generating 2500 transactions [0, 0, 0] 2024-04-04T10:54:39.535000Z TestFramework (INFO): Iteration 1, generating 2500 transactions [0, 0, 0] 2024-04-04T12:00:57.244000Z TestFramework (INFO): Iteration 2, generating 2500 transactions [1, 0, 0] 2024-04-04T13:07:27.698000Z TestFramework (INFO): Iteration 3, generating 2500 transactions [1, 0, 0] 2024-04-04T14:14:32.395000Z TestFramework (INFO): Verifying utxo hash matches for all nodes 2024-04-04T14:14:32.482000Z TestFramework (INFO): Restarted nodes: [2, 0, 0]; crashes on restart: 1 2024-04-04T14:14:32.482000Z TestFramework (WARNING): Node 1 never crashed during utxo flush! 2024-04-04T14:14:32.482000Z TestFramework (WARNING): Node 2 never crashed during utxo flush! 2024-04-04T14:14:32.536000Z TestFramework (INFO): Stopping nodes 2024-04-04T14:14:32.840000Z TestFramework (INFO): Cleaning up /tmp/bitcoin_func_test_dybbv4yh on exit 2024-04-04T14:14:32.840000Z TestFramework (INFO): Tests successful

approx 6 hours
@jamesdorfman
Copy link
Collaborator

jamesdorfman commented Apr 5, 2024 •
edited
Loading

Hmm... seems to still be happening though:

[james@elements-workspace elements]$ ./test/functional/feature_dbcrash.py 2024-04-04T20:07:55.335000Z TestFramework (INFO): Initializing test directory /tmp/bitcoin_func_test_e8dp9mz5 2024-04-04T21:11:27.390000Z TestFramework (INFO): Prepped 5001 utxo entries 2024-04-04T21:12:03.369000Z TestFramework (INFO): Iteration 0, generating 2500 transactions [1, 0, 0] 2024-04-04T22:47:36.411000Z TestFramework (INFO): Iteration 1, generating 2500 transactions [1, 0, 0] 2024-04-05T00:22:26.239000Z TestFramework (INFO): Iteration 2, generating 2500 transactions [2, 0, 0] 2024-04-05T01:57:35.443000Z TestFramework (INFO): Iteration 3, generating 2500 transactions [2, 0, 0] 2024-04-05T03:33:23.923000Z TestFramework (INFO): Verifying utxo hash matches for all nodes 2024-04-05T03:33:24.848000Z TestFramework (INFO): Restarted nodes: [3, 0, 0]; crashes on restart: 0 2024-04-05T03:33:24.848000Z TestFramework (ERROR): Assertion failed Traceback (most recent call last): File "/home/james/elements/test/functional/test_framework/test_framework.py", line 132, in main self.run_test() File "/home/james/elements/./test/functional/feature_dbcrash.py", line 285, in run_test assert self.crashed_on_restart > 0 ^^^^^^^^^^^^^^^^^^^^^^^^^^^ AssertionError 2024-04-05T03:33:24.913000Z TestFramework (INFO): Stopping nodes 2024-04-05T03:33:25.220000Z TestFramework (WARNING): Not cleaning up dir /tmp/bitcoin_func_test_e8dp9mz5 2024-04-05T03:33:25.220000Z TestFramework (ERROR): Test failed. Test logging available at /tmp/bitcoin_func_test_e8dp9mz5/test_framework.log 2024-04-05T03:33:25.220000Z TestFramework (ERROR): 2024-04-05T03:33:25.221000Z TestFramework (ERROR): Hint: Call /home/james/elements/test/functional/combine_logs.py '/tmp/bitcoin_func_test_e8dp9mz5' to consolidate all logs 2024-04-05T03:33:25.221000Z TestFramework (ERROR): 2024-04-05T03:33:25.221000Z TestFramework (ERROR): If this failure happened unexpectedly or intermittently, please file a bug and provide a link or upload of the combined log. 2024-04-05T03:33:25.221000Z TestFramework (ERROR): https://github.com/ElementsProject/elements/issues 2024-04-05T03:33:25.221000Z TestFramework (ERROR): [james@elements-workspace elements]$ git log commit 1ed7f84f05b54234439fc85436bd2b8854353812 (HEAD, origin/issues-1297)
@delta1
Copy link
Member Author

delta1 commented Apr 5, 2024 •
edited
Loading

strange, worked for me again. i'm going to bump up the iteration and run it again.

2024-04-04T15:10:11.209000Z TestFramework (INFO): Initializing test directory /tmp/bitcoin_func_test_1c04byhm 2024-04-04T15:53:58.589000Z TestFramework (INFO): Prepped 5001 utxo entries 2024-04-04T15:54:27.777000Z TestFramework (INFO): Iteration 0, generating 2500 transactions [0, 0, 0] 2024-04-04T17:00:24.418000Z TestFramework (INFO): Iteration 1, generating 2500 transactions [0, 0, 0] 2024-04-04T18:06:32.628000Z TestFramework (INFO): Iteration 2, generating 2500 transactions [0, 0, 0] 2024-04-04T19:12:57.430000Z TestFramework (INFO): Iteration 3, generating 2500 transactions [1, 1, 0] 2024-04-04T20:19:31.617000Z TestFramework (INFO): Verifying utxo hash matches for all nodes 2024-04-04T20:19:32.528000Z TestFramework (INFO): Restarted nodes: [1, 1, 0]; crashes on restart: 2 2024-04-04T20:19:32.528000Z TestFramework (WARNING): Node 2 never crashed during utxo flush! 2024-04-04T20:19:32.585000Z TestFramework (INFO): Stopping nodes 2024-04-04T20:19:32.889000Z TestFramework (INFO): Cleaning up /tmp/bitcoin_func_test_1c04byhm on exit 2024-04-04T20:19:32.889000Z TestFramework (INFO): Tests successful
@delta1
Copy link
Member Author

delta1 commented Apr 5, 2024 •
edited
Loading

5 iterations

> time test/functional/feature_dbcrash.py 2024-04-05T07:05:16.203000Z TestFramework (INFO): Initializing test directory /tmp/bitcoin_func_test_8_zlhs9q 2024-04-05T07:48:56.506000Z TestFramework (INFO): Prepped 5001 utxo entries 2024-04-05T07:49:24.107000Z TestFramework (INFO): Iteration 0, generating 2500 transactions [0, 0, 0] 2024-04-05T08:55:19.380000Z TestFramework (INFO): Iteration 1, generating 2500 transactions [0, 0, 0] 2024-04-05T10:01:41.702000Z TestFramework (INFO): Iteration 2, generating 2500 transactions [1, 0, 0] 2024-04-05T11:08:10.640000Z TestFramework (INFO): Iteration 3, generating 2500 transactions [1, 0, 0] 2024-04-05T12:15:03.874000Z TestFramework (INFO): Iteration 4, generating 2500 transactions [2, 0, 0] 2024-04-05T13:21:56.336000Z TestFramework (INFO): Verifying utxo hash matches for all nodes 2024-04-05T13:21:58.731000Z TestFramework (INFO): Restarted nodes: [3, 1, 0]; crashes on restart: 2 2024-04-05T13:21:58.731000Z TestFramework (WARNING): Node 2 never crashed during utxo flush! 2024-04-05T13:21:58.791000Z TestFramework (INFO): Stopping nodes 2024-04-05T13:21:59.045000Z TestFramework (INFO): Cleaning up /tmp/bitcoin_func_test_8_zlhs9q on exit 2024-04-05T13:21:59.045000Z TestFramework (INFO): Tests successful test/functional/feature_dbcrash.py 22436.98s user 118.79s system 99% cpu 6:16:42.95 total

and

> time test/functional/feature_dbcrash.py 2024-04-05T14:43:53.223000Z TestFramework (INFO): Initializing test directory /tmp/bitcoin_func_test_y3m7o8tu 2024-04-05T15:27:36.512000Z TestFramework (INFO): Prepped 5001 utxo entries 2024-04-05T15:28:06.834000Z TestFramework (INFO): Iteration 0, generating 2500 transactions [1, 0, 0] 2024-04-05T16:33:52.073000Z TestFramework (INFO): Iteration 1, generating 2500 transactions [1, 0, 0] 2024-04-05T17:40:09.012000Z TestFramework (INFO): Iteration 2, generating 2500 transactions [2, 0, 0] 2024-04-05T18:46:39.010000Z TestFramework (INFO): Iteration 3, generating 2500 transactions [3, 1, 0] 2024-04-05T19:53:15.120000Z TestFramework (INFO): Iteration 4, generating 2500 transactions [4, 1, 0] 2024-04-05T21:00:19.098000Z TestFramework (INFO): Verifying utxo hash matches for all nodes 2024-04-05T21:00:19.222000Z TestFramework (INFO): Restarted nodes: [4, 2, 0]; crashes on restart: 1 2024-04-05T21:00:19.222000Z TestFramework (WARNING): Node 2 never crashed during utxo flush! 2024-04-05T21:00:19.280000Z TestFramework (INFO): Stopping nodes 2024-04-05T21:00:19.734000Z TestFramework (INFO): Cleaning up /tmp/bitcoin_func_test_y3m7o8tu on exit 2024-04-05T21:00:19.734000Z TestFramework (INFO): Tests successful test/functional/feature_dbcrash.py 22428.39s user 119.89s system 99% cpu 6:16:26.63 total
@delta1
Copy link
Member Author

delta1 commented Apr 7, 2024

@jamesdorfman had a similar failure. bumped the iterations to 6 and will run a number of tests
@whitslack
Copy link

whitslack commented Apr 7, 2024

@delta1: It looks as though you're trying to tune a probabilistic test. You're never going to find a number that gives the desired result on every machine from the slowest single-core SoC to the fastest NUMA supercomputer. Instead, the test case should be fixed so that it is deterministic. If that is not possible, then the test case should be discarded.
@delta1
Copy link
Member Author

delta1 commented Apr 8, 2024

I suspect there’s a threshold that works reliably like the Bitcoin test.
@delta1 delta1 force-pushed the issues-1297 branch 2 times, most recently from 9dc2afe to 98ad8c1 Compare April 9, 2024 07:35
JBetz added a commit to SequentiaSEQ/SEQ-Core-Elements that referenced this pull request May 2, 2024
@JBetz
Apply fixes to fee estimation functional test from ElementsProject/el…
bcf9e69 
…ements#1298
JBetz added a commit to SequentiaSEQ/SEQ-Core-Elements that referenced this pull request May 3, 2024
@JBetz
Interpret fee rates in reference fee unit (#21)
b7d3729 
* Introduce CValue for distinguishing amounts from values * Remove random character * Define insertion operator for CValue * Move CValue to separate module * Remove consensus changes * Remove CValue usage for fee rates, restrict to mempool * More cleanup * Use CValue for descendant and ancestor fee updates * More mempool fixes * Remove unused include * Fix variable shadowing * More conversions * Factor in fee asset when validating package fees * More conversions * Convert dust relay fee from RFU * More cleanup * Cleanup * Don't factor in asset in when calculating dust threshold * Convert package fees * Return CValue from CalculateExchangeValue * Use CValue for reverse exchange rate conversion * Update method documentation * Rename currency conversion methods * Fix typo * Add justification for RFU to comments * Use accessor instead of public instance variable unpacking CValue * Use GetValue() in fee rate conversion * Only apply dust check to outputs denominated in same asset as fee * Interpret coin selection effective feerate in RFU * Change GetModifiedFee() method to use RFU for summing with ancestor and descendant transactions * Add functional test for "fee_rate" parameter in fundrawtransaction RPC and -mintxfee node configuration parameter * Normalize fees to RFU during fee estimation * Lint fixes * More linting fixes * More linting * Apply method renams to fee amount conversions * Add any asset fee rates test to test runner * More amount conversion fixes * Add compiler switch for currency constants * Add new configuration flag to documentation * Apply fixes to fee estimation functional test from ElementsProject/elements#1298 * Add tests for paytxfee parameter * Cleanup * Fix compiler flag * Add tests for blockmintxfee * Remove whitespace * Add comments clarifying what's being demonstrated * Remove nFeeValue, recompute as needed * Revert "Remove nFeeValue, recompute as needed" This reverts commit ab3a67b. * Move No Coin configuration documentation to separate line * Add constant for full name of currency atom * Fix constant reference * Fix typo
JBetz added a commit to SequentiaSEQ/SEQ-Core-Elements that referenced this pull request May 3, 2024
@JBetz
Interpret fee rates in reference fee unit (#21)
5f8dadf 
* Introduce CValue for distinguishing amounts from values * Remove random character * Define insertion operator for CValue * Move CValue to separate module * Remove consensus changes * Remove CValue usage for fee rates, restrict to mempool * More cleanup * Use CValue for descendant and ancestor fee updates * More mempool fixes * Remove unused include * Fix variable shadowing * More conversions * Factor in fee asset when validating package fees * More conversions * Convert dust relay fee from RFU * More cleanup * Cleanup * Don't factor in asset in when calculating dust threshold * Convert package fees * Return CValue from CalculateExchangeValue * Use CValue for reverse exchange rate conversion * Update method documentation * Rename currency conversion methods * Fix typo * Add justification for RFU to comments * Use accessor instead of public instance variable unpacking CValue * Use GetValue() in fee rate conversion * Only apply dust check to outputs denominated in same asset as fee * Interpret coin selection effective feerate in RFU * Change GetModifiedFee() method to use RFU for summing with ancestor and descendant transactions * Add functional test for "fee_rate" parameter in fundrawtransaction RPC and -mintxfee node configuration parameter * Normalize fees to RFU during fee estimation * Lint fixes * More linting fixes * More linting * Apply method renams to fee amount conversions * Add any asset fee rates test to test runner * More amount conversion fixes * Add compiler switch for currency constants * Add new configuration flag to documentation * Apply fixes to fee estimation functional test from ElementsProject/elements#1298 * Add tests for paytxfee parameter * Cleanup * Fix compiler flag * Add tests for blockmintxfee * Remove whitespace * Add comments clarifying what's being demonstrated * Remove nFeeValue, recompute as needed * Revert "Remove nFeeValue, recompute as needed" This reverts commit ab3a67b. * Move No Coin configuration documentation to separate line * Add constant for full name of currency atom * Fix constant reference * Fix typo
JBetz added a commit to SequentiaSEQ/SEQ-Core-Elements that referenced this pull request May 3, 2024
@JBetz
Interpret fee rates in reference fee unit (#21)
b913bde 
* Introduce CValue for distinguishing amounts from values * Remove random character * Define insertion operator for CValue * Move CValue to separate module * Remove consensus changes * Remove CValue usage for fee rates, restrict to mempool * More cleanup * Use CValue for descendant and ancestor fee updates * More mempool fixes * Remove unused include * Fix variable shadowing * More conversions * Factor in fee asset when validating package fees * More conversions * Convert dust relay fee from RFU * More cleanup * Cleanup * Don't factor in asset in when calculating dust threshold * Convert package fees * Return CValue from CalculateExchangeValue * Use CValue for reverse exchange rate conversion * Update method documentation * Rename currency conversion methods * Fix typo * Add justification for RFU to comments * Use accessor instead of public instance variable unpacking CValue * Use GetValue() in fee rate conversion * Only apply dust check to outputs denominated in same asset as fee * Interpret coin selection effective feerate in RFU * Change GetModifiedFee() method to use RFU for summing with ancestor and descendant transactions * Add functional test for "fee_rate" parameter in fundrawtransaction RPC and -mintxfee node configuration parameter * Normalize fees to RFU during fee estimation * Lint fixes * More linting fixes * More linting * Apply method renams to fee amount conversions * Add any asset fee rates test to test runner * More amount conversion fixes * Add compiler switch for currency constants * Add new configuration flag to documentation * Apply fixes to fee estimation functional test from ElementsProject/elements#1298 * Add tests for paytxfee parameter * Cleanup * Fix compiler flag * Add tests for blockmintxfee * Remove whitespace * Add comments clarifying what's being demonstrated * Remove nFeeValue, recompute as needed * Revert "Remove nFeeValue, recompute as needed" This reverts commit ab3a67b. * Move No Coin configuration documentation to separate line * Add constant for full name of currency atom * Fix constant reference * Fix typo
Mixa84 pushed a commit to SequentiaSEQ/SEQ-Core-Elements that referenced this pull request May 6, 2024
@JBetz @Mixa84
Interpret fee rates in reference fee unit (#21)
8ffe5aa 
* Introduce CValue for distinguishing amounts from values * Remove random character * Define insertion operator for CValue * Move CValue to separate module * Remove consensus changes * Remove CValue usage for fee rates, restrict to mempool * More cleanup * Use CValue for descendant and ancestor fee updates * More mempool fixes * Remove unused include * Fix variable shadowing * More conversions * Factor in fee asset when validating package fees * More conversions * Convert dust relay fee from RFU * More cleanup * Cleanup * Don't factor in asset in when calculating dust threshold * Convert package fees * Return CValue from CalculateExchangeValue * Use CValue for reverse exchange rate conversion * Update method documentation * Rename currency conversion methods * Fix typo * Add justification for RFU to comments * Use accessor instead of public instance variable unpacking CValue * Use GetValue() in fee rate conversion * Only apply dust check to outputs denominated in same asset as fee * Interpret coin selection effective feerate in RFU * Change GetModifiedFee() method to use RFU for summing with ancestor and descendant transactions * Add functional test for "fee_rate" parameter in fundrawtransaction RPC and -mintxfee node configuration parameter * Normalize fees to RFU during fee estimation * Lint fixes * More linting fixes * More linting * Apply method renams to fee amount conversions * Add any asset fee rates test to test runner * More amount conversion fixes * Add compiler switch for currency constants * Add new configuration flag to documentation * Apply fixes to fee estimation functional test from ElementsProject/elements#1298 * Add tests for paytxfee parameter * Cleanup * Fix compiler flag * Add tests for blockmintxfee * Remove whitespace * Add comments clarifying what's being demonstrated * Remove nFeeValue, recompute as needed * Revert "Remove nFeeValue, recompute as needed" This reverts commit ab3a67b. * Move No Coin configuration documentation to separate line * Add constant for full name of currency atom * Fix constant reference * Fix typo
@delta1 delta1 mentioned this pull request Sep 17, 2024
Closed
@psgreco psgreco mentioned this pull request Jun 24, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

3 participants

@delta1 @jamesdorfman @whitslack