[3.11] gh-135034: Normalize link targets in tarfile, add os.path.realpath(strict='allow_missing') (GH-135037)

[Yhg1s]

Addresses CVEs 2024-12718, 2025-4138, 2025-4330, and 2025-4517.
(cherry picked from commit 3612d8f)

Co-authored-by: Łukasz Langa lukasz@langa.pl
Co-authored-by: Petr Viktorin encukou@gmail.com
Co-authored-by: Seth Michael Larson seth@python.org
Co-authored-by: Adam Turner 9087854+AA-Turner@users.noreply.github.com
Co-authored-by: Serhiy Storchaka storchaka@gmail.com

• Issue: Multiple tarfile extraction filter bypasses (filter="tar"/filter="data") #135034

---

📚 Documentation preview 📚: https://cpython-previews--135068.org.readthedocs.build/

[kulikjak]

After this update, I am getting the following test failure:

====================================================================== ERROR: test_realpath_limit_attack (test.test_tarfile.TestExtractionFilters.test_realpath_limit_attack) [fully_trusted] ---------------------------------------------------------------------- Traceback (most recent call last): File "/builds/jkulik/python-3.13.4-3.11.13-3.9.23/components/python/python311/Python-3.11.13/Lib/test/test_tarfile.py", line 3669, in test_realpath_limit_attack with (self.subTest('fully_trusted'), File "/builds/jkulik/python-3.13.4-3.11.13-3.9.23/components/python/python311/Python-3.11.13/Lib/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/builds/jkulik/python-3.13.4-3.11.13-3.9.23/components/python/python311/Python-3.11.13/Lib/test/test_tarfile.py", line 3458, in check_context self.expected_paths = set(self.outerdir.glob('**/*')) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/builds/jkulik/python-3.13.4-3.11.13-3.9.23/components/python/python311/Python-3.11.13/Lib/pathlib.py", line 958, in glob for p in selector.select_from(self): File "/builds/jkulik/python-3.13.4-3.11.13-3.9.23/components/python/python311/Python-3.11.13/Lib/pathlib.py", line 411, in _select_from for starting_point in self._iterate_directories(parent_path, is_dir, scandir): File "/builds/jkulik/python-3.13.4-3.11.13-3.9.23/components/python/python311/Python-3.11.13/Lib/pathlib.py", line 401, in _iterate_directories for p in self._iterate_directories(path, is_dir, scandir): File "/builds/jkulik/python-3.13.4-3.11.13-3.9.23/components/python/python311/Python-3.11.13/Lib/pathlib.py", line 401, in _iterate_directories for p in self._iterate_directories(path, is_dir, scandir): File "/builds/jkulik/python-3.13.4-3.11.13-3.9.23/components/python/python311/Python-3.11.13/Lib/pathlib.py", line 401, in _iterate_directories for p in self._iterate_directories(path, is_dir, scandir): [Previous line repeated 14 more times] File "/builds/jkulik/python-3.13.4-3.11.13-3.9.23/components/python/python311/Python-3.11.13/Lib/pathlib.py", line 395, in _iterate_directories entry_is_dir = entry.is_dir(follow_symlinks=False) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ OSError: [Errno 78] File name too long: '/builds/jkulik/python-3.13.4-3.11.13-3.9.23/components/python/python311/build/amd64/build/test_python_22904æ/@test_22904_tmpæ-tardir/outerdir/dest/ddddddddddddddddddd dddddddddddddddddddddddddddddddd/ddddddddddddddddddddddddddddddddddddddddddddddddddd/ddddddddddddddddddddddddddddddddddddddddddddddddddd/ddddddddddddddddddddddddddddddddddddddddddddddddddd/dddddddddddddddddd ddddddddddddddddddddddddddddddddd/ddddddddddddddddddddddddddddddddddddddddddddddddddd/ddddddddddddddddddddddddddddddddddddddddddddddddddd/ddddddddddddddddddddddddddddddddddddddddddddddddddd/ddddddddddddddddd dddddddddddddddddddddddddddddddddd/ddddddddddddddddddddddddddddddddddddddddddddddddddd/ddddddddddddddddddddddddddddddddddddddddddddddddddd/ddddddddddddddddddddddddddddddddddddddddddddddddddd/dddddddddddddddd ddddddddddddddddddddddddddddddddddd/ddddddddddddddddddddddddddddddddddddddddddddddddddd/ddddddddddddddddddddddddddddddddddddddddddddddddddd/ddddddddddddddddddddddddddddddddddddddddddddddddddd/lllllllllllllll lllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllll llllllllllllllllllllllllllllllll'

Interestingly, it's just 3.11.13 that's failing - both 3.9.23 and 3.13.4 (and all other supported versions we are running in our internal buildbot) pass. 3.13 has a very different pathlib implementation, but 3.9 is pretty similar so I investigated the differences, and when I print the entires from here (_iterate_directories 3.9 / 3.11):

 with scandir(parent_path) as scandir_it: entries = list(scandir_it) 

I get very different results:
3.9:

[<DirEntry 'newfile'>, <DirEntry 'flag'>, <DirEntry 'dest'>] [<DirEntry 'a'>, <DirEntry 'flaglink'>, <DirEntry 'escape'>] 

3.11:

[<DirEntry 'dest'>, <DirEntry 'flag'>, <DirEntry 'newfile'>] [<DirEntry 'a'>, <DirEntry 'escape'>, <DirEntry 'flaglink'>, <DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>] [<DirEntry 'b'>, <DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>] [<DirEntry 'c'>, <DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>] [<DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>, <DirEntry 'd'>] [<DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>, <DirEntry 'e'>] [<DirEntry 'f'>, <DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>] [<DirEntry 'g'>, <DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>] [<DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>, <DirEntry 'h'>] [<DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>, <DirEntry 'i'>] [<DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>, <DirEntry 'j'>] [<DirEntry 'k'>, <DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>] [<DirEntry 'l'>, <DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>] [<DirEntry 'm'>, <DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>] [<DirEntry 'n'>, <DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>] [<DirEntry 'o'>, <DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>] [<DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>, <DirEntry 'p'>] [<DirEntry 'lllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllll lllllllllllllllllllllllllllllllllllllllllllllllllllllllllll'>] 

Also, the scandir argument is different:
3.9: <built-in function scandir>
3.11: <function Path._scandir at 0x7fed9bd923e0>
so I guess that those implementations behave differently (the 3.11 one returns much more entires), which causes the failure?

This is on Oracle Solaris.

[encukou]

Ah! We got the same test failure in 3.10, so I hotfixed it there. Try applying the hack to 3.11: dff62a1

It's just a test failure: if the filter is fully_trusted, this archive unpacks to a very interesting directory tree -- one that this version of Path.glob can't handle. And we call glob in the tests to get data for the assertions.

[kulikjak]

Oh, that fixed it. Thanks!

(i didn't realize that this difference between 3.9 and 3.11 patch is what causes the issue)

[DimNik9]

Hello, sorry to ask directly here but i haven't understood how 3.10 & 3.11 are affected by these CVEs, since the description of each CVE states that ONLY versions after 3.12 are affected, since the extraction filters were first introduced in 3.12
Thank you in advance

[encukou]

yeah, looks like the CVE text needs an update :(
cc @sethmlarson
The feature was backported to 3.8.17, 3.9.17, 3.10.12, 3.11.4

[sethmlarson]

@encukou Gotcha, I didn't realize that filtering had been backported, I'll update the prose description in each document. The affectedness of the CVEs is correct thankfully :)

[sethmlarson]

Updated the CVE records and sent a correction to security-announce@python.org.