Bug report

Bug description:

I am suspicious about the purpose of the test_compress_locking test in test_zstd.py, and actually found a race condition in it, when using a freethreaded build.

From my understanding, it is doing the following thing:

1. allocate a ZstdCompressor instance
2. on each of 8 threads:
   1. call .compress method with the string b'a'*16384
   2. check if the result is empty
   3. .append the result to a list common to all threads
3. (more steps here outside of the threading part)
4. check if the content of the list is the same as expected

Now, because the input of .compress method is the same on all threads, here is the output of that method:

• if it is the first call, then the zstd header will be added, so b'(\xb5/\xfd\x00XL\x00\x00\x10aa\x01\x00\xfb\x9f\x07X'
• else (for all 7 others): b'\x02\x00\x02a'

However, there is a logic issue: the thread that calls first the .compress method in step 2.i. may not be the same thread that calls the .append in step 2.iii.: this is a race condition. If that happens, the result is not the same as expected and the test fails.

For some reason, this happens very infrequently, but it does (at least on free-threading build), here on attempt 307:

$ for i in `seq 1000`; do echo $i; ./python -m test.test_zstd -vv -k test_compress_locking || break; done <<<redacted>>> 306 test_compress_locking (__main__.FreeThreadingMethodTests.test_compress_locking) ... FAIL ====================================================================== FAIL: test_compress_locking (__main__.FreeThreadingMethodTests.test_compress_locking) ---------------------------------------------------------------------- Traceback (most recent call last): File "/redacted/cpython/Lib/test/support/threading_helper.py", line 66, in decorator return func(*args) File "/redacted/cpython/Lib/test/test_zstd.py", line 2704, in test_compress_locking self.assertEqual(expected, actual) ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^ AssertionError: b'(\xb5/\xfd\x00XL\x00\x00\x10aa\x01\x00\xf[109 chars]\x00' != b'\x02\x00\x02a\x02\x00\x02a(\xb5/\xfd\x00X[109 chars]\x00' ---------------------------------------------------------------------- Ran 1 test in 0.069s FAILED (failures=1) 

So here is my question: what is the test expected to be testing exactly? How can we rewrite the test to still perform the expected test, while at the same time not having the race condition mentioned above?

Also it may be worth looking in other tests of FreeThreadingMethodTests to see if they are impacted.

Tested on main (0240ef4) with a free-threading build, but should be the same in 3.14.0b3.
Please add to Compression issues (view)
Paging @emmatyping

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Linked PRs

• gh-136394: Fix race condition in test_zstd #136432
• gh-136394: Make zstd freethreaded compression test order independent #136444