Unable to retrieve resources from a namespace package #68
Description
In GitLab by @jaraco on Nov 2, 2018, 02:48
Attempting to retrieve resources from a namespace package fails.
draft $ mkdir foo draft $ touch foo/bar.txt draft $ rwt importlib_resources Collecting importlib_resources Using cached https://files.pythonhosted.org/packages/2f/f7/b4aa02cdd3ee7ebba375969d77c00826aa15c5db84247d23c89522dccbfa/importlib_resources-1.0.2-py2.py3-none-any.whl Installing collected packages: importlib-resources Successfully installed importlib-resources-1.0.2 Python 3.7.1 (v3.7.1:260ec2c36a, Oct 20 2018, 03:13:28) [Clang 6.0 (clang-600.0.57)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import importlib_resources >>> importlib_resources.read_text(__import__('foo'), 'bar.txt') Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/importlib/resources.py", line 169, in read_text with open_text(package, resource, encoding, errors) as fp: File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/importlib/resources.py", line 126, in open_text _check_location(package) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/importlib/resources.py", line 82, in _check_location raise FileNotFoundError(f'Package has no location {package!r}') FileNotFoundError: Package has no location <module 'foo' (namespace)> I see an obvious problem here - that a namespace package can have more than one base path, so it has no single location. But it does have a location... and pkg_resources lets one load resources from namespace package:
draft $ cat > foo/__init__.py import pkg_resources; pkg_resources.declare_namespace('foo') draft $ python Python 3.7.1 (v3.7.1:260ec2c36a, Oct 20 2018, 03:13:28) [Clang 6.0 (clang-600.0.57)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import pkg_resources >>> pkg_resources.resource_stream('foo', 'bar.txt') <_io.BufferedReader name='/Users/jaraco/draft/foo/bar.txt'> pkg_resources doesn't succeed with a PEP 420 namespace package:
$ rm foo/__init__.py >>> pkg_resources.resource_stream('foo', 'bar.txt') Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pkg_resources/__init__.py", line 1145, in resource_stream return get_provider(package_or_requirement).get_resource_stream( File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pkg_resources/__init__.py", line 359, in get_provider return _find_adapter(_provider_factories, loader)(module) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pkg_resources/__init__.py", line 1387, in __init__ self.module_path = os.path.dirname(getattr(module, '__file__', '')) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/posixpath.py", line 156, in dirname p = os.fspath(p) TypeError: expected str, bytes or os.PathLike object, not NoneType But even in that situation, it does allow for loading resources for a module within a PEP 420 namespace package:
draft $ touch foo/mod.py draft $ tree . └── foo ├── bar.txt └── mod.py draft $ python Python 3.7.1 (v3.7.1:260ec2c36a, Oct 20 2018, 03:13:28) [Clang 6.0 (clang-600.0.57)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import pkg_resources >>> pkg_resources.resource_stream('foo.mod', 'bar.txt') <_io.BufferedReader name='/Users/jaraco/draft/foo/bar.txt'> This issue sort-of relates to #60, but is more serious because it seems there's no input that importlib_resources can accept to load resources from a namespace package.
The issue emerged in pmxbot when I tried to convert the package from a (deprecated) pkg_resources-style namespace package to a PEP 420 namespace package. The code failed at this line when it tried to load the phrases.
It seems to me (without looking at the code) it should be straightforward to support loading resources from namespace packages, following the same logic that importlib follows to import a module from that package.
The only other option I see is to force packages to rewrite their packages to only put resources in non-namespace packages, which is a bit of an imposition and unintuitive constraint.