AIX: pip download package does not work since 21.2 #10858
Description
Description
Recently got started with needing to update a number of packages - and ran into a problem that I could not download ansible-base (so I suspect it is an issue with any packages that are not pure Python).
So, rolled back pip to a much older version (20.2.4) and all was okay.
In increments I updated pip and 21.1.3 was the last version that worked as expected (as far as download is concerned, have not tried anything else).
Where I think the regression occurred
aixtools@x064:[/data/prj/python/git/pip]git diff 21.1.3 21.2.1 -- ./src/pip/_internal/utils/unpacking.py diff --git a/src/pip/_internal/utils/unpacking.py b/src/pip/_internal/utils/unpacking.py index 44ac47535..bffb3cd65 100644 --- a/src/pip/_internal/utils/unpacking.py +++ b/src/pip/_internal/utils/unpacking.py @@ -178,7 +178,7 @@ def untar_file(filename, location): filename, ) mode = "r:*" - tar = tarfile.open(filename, mode) + tar = tarfile.open(filename, mode, encoding="utf-8") try: leading = has_leading_dir([member.name for member in tar.getmembers()]) for member in tar.getmembers(): Expected behavior
- Note: I get the same problem as above when trying to download ansible-base==2.10.16, but the
ansibleI have been using for two years is based on 2.10.1 - so I tried that version again. - I saw you are withdrawing support for py36 - so, please note, this is not a request for py36 support. I first saw this on Python3-9 which is what I wanted to update.
py36is only being used because that is known to be working - and I was looking for when the regression appeared.
(py360) aixtools@x064:[/home/aixtools/download/py360]pip download ansible-base==2.10.1 Collecting ansible-base==2.10.1 Using cached ansible-base-2.10.1.tar.gz (6.0 MB) Collecting jinja2 Using cached Jinja2-3.0.3-py3-none-any.whl (133 kB) Collecting PyYAML Using cached PyYAML-6.0.tar.gz (124 kB) Installing build dependencies ... done Getting requirements to build wheel ... done Preparing wheel metadata ... done Collecting cryptography Using cached cryptography-36.0.1.tar.gz (572 kB) Installing build dependencies ... done Getting requirements to build wheel ... done Preparing wheel metadata ... done Collecting packaging Downloading packaging-21.3-py3-none-any.whl (40 kB) |################################| 40 kB 44 kB/s Collecting cffi>=1.12 Using cached cffi-1.15.0.tar.gz (484 kB) Collecting pycparser Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB) Collecting MarkupSafe>=2.0 Downloading MarkupSafe-2.0.1.tar.gz (18 kB) Collecting pyparsing!=3.0.5,>=2.0.2 Downloading pyparsing-3.0.7-py3-none-any.whl (98 kB) |################################| 98 kB 286 kB/s Saved ./ansible-base-2.10.1.tar.gz Saved ./cryptography-36.0.1.tar.gz Saved ./cffi-1.15.0.tar.gz Saved ./Jinja2-3.0.3-py3-none-any.whl Saved ./MarkupSafe-2.0.1.tar.gz Saved ./packaging-21.3-py3-none-any.whl Saved ./pyparsing-3.0.7-py3-none-any.whl Saved ./pycparser-2.21-py2.py3-none-any.whl Saved ./PyYAML-6.0.tar.gz pip version
21.3.1, 21.2.4, 21.2
Python version
3.6, 3.9
OS
AIX
How to Reproduce
- System Admin Installs Python3-9 (or 3-6) including virtualenv
- Following steps as regular (no root powers)
- virtualenv py360
- mkdir -p downloads/py360
- . py360/bin/activate
- cd downloads/py360
- pip3 download ansible-base==2.10.1
Output
- Example: (py360) aixtools@x064:[/home/aixtools/download/py360]pip install pip --upgrade Requirement already satisfied: pip in /home/aixtools/py360/lib/python3.6/site-packages (21.1.3) Collecting pip Using cached pip-21.3.1-py3-none-any.whl (1.7 MB) Installing collected packages: pip Attempting uninstall: pip Found existing installation: pip 21.1.3 Uninstalling pip-21.1.3: Successfully uninstalled pip-21.1.3 Successfully installed pip-21.3.1 (py360) aixtools@x064:[/home/aixtools/download/py360]pip download ansible-base==2.10.1 Collecting ansible-base==2.10.1 File was already downloaded /home/aixtools/download/py360/ansible-base-2.10.1.tar.gz ERROR: Exception: Traceback (most recent call last): File "/home/aixtools/py360/lib/python3.6/site-packages/pip/_internal/cli/base_command.py", line 164, in exc_logging_wrapper status = run_func(*args) File "/home/aixtools/py360/lib/python3.6/site-packages/pip/_internal/cli/req_command.py", line 205, in wrapper return func(self, options, args) File "/home/aixtools/py360/lib/python3.6/site-packages/pip/_internal/commands/download.py", line 128, in run requirement_set = resolver.resolve(reqs, check_supported_wheels=True) File "/home/aixtools/py360/lib/python3.6/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 93, in resolve collected.requirements, max_rounds=try_to_avoid_resolution_too_deep File "/home/aixtools/py360/lib/python3.6/site-packages/pip/_vendor/resolvelib/resolvers.py", line 482, in resolve state = resolution.resolve(requirements, max_rounds=max_rounds) File "/home/aixtools/py360/lib/python3.6/site-packages/pip/_vendor/resolvelib/resolvers.py", line 349, in resolve self._add_to_criteria(self.state.criteria, r, parent=None) File "/home/aixtools/py360/lib/python3.6/site-packages/pip/_vendor/resolvelib/resolvers.py", line 173, in _add_to_criteria if not criterion.candidates: File "/home/aixtools/py360/lib/python3.6/site-packages/pip/_vendor/resolvelib/structs.py", line 151, in __bool__ return bool(self._sequence) File "/home/aixtools/py360/lib/python3.6/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 155, in __bool__ return any(self) File "/home/aixtools/py360/lib/python3.6/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 143, in <genexpr> return (c for c in iterator if id(c) not in self._incompatible_ids) File "/home/aixtools/py360/lib/python3.6/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 47, in _iter_built candidate = func() File "/home/aixtools/py360/lib/python3.6/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 206, in _make_candidate_from_link version=version, File "/home/aixtools/py360/lib/python3.6/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 287, in __init__ version=version, File "/home/aixtools/py360/lib/python3.6/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 156, in __init__ self.dist = self._prepare() File "/home/aixtools/py360/lib/python3.6/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 225, in _prepare dist = self._prepare_distribution() File "/home/aixtools/py360/lib/python3.6/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 292, in _prepare_distribution return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True) File "/home/aixtools/py360/lib/python3.6/site-packages/pip/_internal/operations/prepare.py", line 482, in prepare_linked_requirement return self._prepare_linked_requirement(req, parallel_builds) File "/home/aixtools/py360/lib/python3.6/site-packages/pip/_internal/operations/prepare.py", line 528, in _prepare_linked_requirement link, req.source_dir, self._download, self.download_dir, hashes File "/home/aixtools/py360/lib/python3.6/site-packages/pip/_internal/operations/prepare.py", line 223, in unpack_url unpack_file(file.path, location, file.content_type) File "/home/aixtools/py360/lib/python3.6/site-packages/pip/_internal/utils/unpacking.py", line 247, in unpack_file untar_file(filename, location) File "/home/aixtools/py360/lib/python3.6/site-packages/pip/_internal/utils/unpacking.py", line 218, in untar_file with open(path, "wb") as destfp: UnicodeEncodeError: 'latin-1' codec can't encode characters in position 138-141: ordinal not in range(256)Code of Conduct
- I agree to follow the PSF Code of Conduct.