Force oldest locks into shard locks due to range limits

[snaury]

Changelog entry

Force oldest (not newest) locks into shard locks due to range limits.

Changelog category

• Improvement

Additional information

Historically datashards forced new locks into whole shard locks when reaching limits on the number of ranges (which incidentally have been the same as the number of locks), which is counter intuitive in the presence of long running read-only transactions that perform joins (and hence lots of point reads), causing very short read-write transactions to repeatedly fail due to unrelated conflicts in the presence of older and wider locks. We prefer forcing oldest locks into shard locks now, which are more likely to break due to conflicts anyway, and may not even cause read-only transactions to fail when they do. Newer locks are now exact (as long as they are small enough), with a significant increase to the maximum number of ranges we support (up to 1M total ranges and 50K per-lock ranges, instead of the previous 16K).

Fixes #10725.

[github-actions]

⚪ 2024-11-06 15:28:29 UTC Pre-commit check linux-x86_64-relwithdebinfo for 6aa6ae1 has started.
⚪ 2024-11-06 15:29:19 UTC Artifacts will be uploaded here
⚪ 2024-11-06 15:33:01 UTC ya make is running...
⚫ 2024-11-06 15:39:02 UTC Check cancelled

[github-actions]

⚪ 2024-11-06 15:31:42 UTC Pre-commit check linux-x86_64-release-asan for 6aa6ae1 has started.
⚪ 2024-11-06 15:31:53 UTC Artifacts will be uploaded here
⚪ 2024-11-06 15:35:04 UTC ya make is running...
⚫ 2024-11-06 15:39:07 UTC Check cancelled

[github-actions]

⚪ 2024-11-06 15:41:42 UTC Pre-commit check linux-x86_64-relwithdebinfo for 4a59623 has started.
⚪ 2024-11-06 15:42:33 UTC Artifacts will be uploaded here
⚫ 2024-11-06 15:43:32 UTC Check cancelled

[github-actions]

⚪ 2024-11-06 15:44:28 UTC Pre-commit check linux-x86_64-relwithdebinfo for 8ee30d7 has started.
⚪ 2024-11-06 15:45:09 UTC Artifacts will be uploaded here
⚪ 2024-11-06 15:48:44 UTC ya make is running...
🟡 2024-11-06 16:37:44 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED?
15328	13819	0	4	1406	99

⚪ 2024-11-06 16:39:13 UTC ya make is running... (failed tests rerun, try 2)
🟢 2024-11-06 16:50:55 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED?
103 (only retried tests)	6	0	0	0	97

🟢 2024-11-06 16:51:01 UTC Build successful.
🟢 2024-11-06 16:51:22 UTC ydbd size 2.8 GiB changed* by +17.0 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: e6ad659	merge: 8ee30d7	diff	diff %
ydbd size	3 044 956 248 Bytes	3 044 973 696 Bytes	+17.0 KiB	+0.001%
ydbd stripped size	481 937 784 Bytes	481 939 192 Bytes	+1.4 KiB	+0.000%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

[github-actions]

⚪ 2024-11-06 15:47:49 UTC Pre-commit check linux-x86_64-release-asan for 8ee30d7 has started.
⚪ 2024-11-06 15:48:01 UTC Artifacts will be uploaded here
⚪ 2024-11-06 15:51:09 UTC ya make is running...
🟡 2024-11-06 17:04:26 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED?
9259	9177	0	34	12	36

🟢 2024-11-06 17:05:15 UTC Build successful.
🟢 2024-11-06 17:05:49 UTC ydbd size 5.7 GiB changed* by +70.3 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: e6ad659	merge: 8ee30d7	diff	diff %
ydbd size	6 161 849 648 Bytes	6 161 921 632 Bytes	+70.3 KiB	+0.001%
ydbd stripped size	1 536 453 712 Bytes	1 536 476 432 Bytes	+22.2 KiB	+0.001%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

[github-actions]

⚪ 2024-11-07 07:52:59 UTC Pre-commit check linux-x86_64-relwithdebinfo for eadec1b has started.
⚪ 2024-11-07 07:53:10 UTC Artifacts will be uploaded here
⚪ 2024-11-07 07:56:20 UTC ya make is running...
🟡 2024-11-07 08:47:33 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED?
15329	13821	0	2	1404	102

⚪ 2024-11-07 08:48:50 UTC ya make is running... (failed tests rerun, try 2)
🟢 2024-11-07 09:00:25 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED?
103 (only retried tests)	7	0	0	0	96

🟢 2024-11-07 09:00:31 UTC Build successful.
🟢 2024-11-07 09:00:51 UTC ydbd size 2.8 GiB changed* by +9.7 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: d6f0772	merge: eadec1b	diff	diff %
ydbd size	3 045 149 688 Bytes	3 045 159 600 Bytes	+9.7 KiB	+0.000%
ydbd stripped size	481 968 696 Bytes	481 970 040 Bytes	+1.3 KiB	+0.000%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

[github-actions]

⚪ 2024-11-07 07:53:10 UTC Pre-commit check linux-x86_64-release-asan for eadec1b has started.
⚪ 2024-11-07 07:53:21 UTC Artifacts will be uploaded here
⚪ 2024-11-07 07:56:30 UTC ya make is running...
🟡 2024-11-07 09:11:19 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED?
9263	9170	0	42	14	37

🟢 2024-11-07 09:12:11 UTC Build successful.
🟢 2024-11-07 09:12:48 UTC ydbd size 5.7 GiB changed* by +47.3 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: d6f0772	merge: eadec1b	diff	diff %
ydbd size	6 162 278 272 Bytes	6 162 326 736 Bytes	+47.3 KiB	+0.001%
ydbd stripped size	1 536 566 320 Bytes	1 536 578 320 Bytes	+11.7 KiB	+0.001%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}