[3.9] bpo-45703: Invalidate _NamespacePath cache on importlib.invalidate_ca… (GH-29384)

[miss-islington]

Consider the following directory structure:

. └── PATH1 └── namespace └── sub1 └── __init__.py 

And both PATH1 and PATH2 in sys path:

$ PYTHONPATH=PATH1:PATH2 python3.11 >>> import namespace >>> import namespace.sub1 >>> namespace.__path__ _NamespacePath(['.../PATH1/namespace']) >>> ... 

While this interpreter still runs, PATH2/namespace/sub2 is created:

. ├── PATH1 │ └── namespace │ └── sub1 │ └── __init__.py └── PATH2 └── namespace └── sub2 └── __init__.py 

The newly created module cannot be imported:

>>> ... >>> namespace.__path__ _NamespacePath(['.../PATH1/namespace']) >>> import namespace.sub2 Traceback (most recent call last): File "<stdin>", line 1, in <module> ModuleNotFoundError: No module named 'namespace.sub2' 

Calling importlib.invalidate_caches() now newly allows to import it:

>>> import importlib >>> importlib.invalidate_caches() >>> namespace.__path__ _NamespacePath(['.../PATH1/namespace']) >>> import namespace.sub2 >>> namespace.__path__ _NamespacePath(['.../PATH1/namespace', '.../PATH2/namespace']) 

This was not previously possible.
(cherry picked from commit ae1965c)

Co-authored-by: Miro Hrončok miro@hroncok.cz

https://bugs.python.org/issue45703

[miss-islington]

@hroncok: Status check is done, and it's a success ✅ .

[miss-islington]

@hroncok: Status check is done, and it's a success ✅ .

[miss-islington]

@hroncok: Status check is done, and it's a success ❌ .

[github-actions]

This PR is stale because it has been open for 30 days with no activity.

[vstinner]

The test failed on Ubuntu:

ERROR: test_invalidate_caches (test.test_importlib.test_namespace_pkgs.SeparatedNamespacePackagesCreatedWhileRunning) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/runner/work/cpython/cpython/Lib/test/test_importlib/test_namespace_pkgs.py", line 158, in test_invalidate_caches import foo.just_created ModuleNotFoundError: No module named 'foo.just_created' 

@hroncok @encukou: Any idea what's going on?

[hroncok]

Yes. This automatic backport will not work. Things are different here. Please close.