Skip to content

Stop installing setuptools and wheel on Python 3.12+ #218

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Aug 26, 2024

Conversation

edmorley
Copy link
Contributor

@edmorley edmorley commented Jul 7, 2024

Currently get-pip.py installs not only pip, but also setuptools and wheel by default, unless the --no-setuptools / --no-wheel (or PIP_NO_SETUPTOOLS / PIP_NO_WHEEL env vars) are used.

This has historically been necessary, however, modern versions of pip will now fallback to pyproject.toml (PEP 517) based builds (which will default to a setuptools backend, and thus automatically install setuptools and wheel in the isolated build environment) if either setuptools is not installed (as of pip 22.1), or if wheel is not installed (as of pip 23.1).

In addition, as of Python 3.12, the stdlib's ensurepip and venv modules no longer install setuptools, and only install pip.

As such, it is now time for get-pip.py to stop installing setuptools and wheel by default on Python 3.12+, in order to:

  • Improve parity with the behaviour of ensurepip / venv on Python 3.12+.
  • Act as another small step towards pyproject.toml / PEP 517 based builds eventually becoming the pip default. (Cross-reference: Default to isolated, pyproject.toml-based builds pip#9175)
  • Allow get-pip.py to focus on its primary responsibility: bootstrapping Pip.

Users can still request that setuptools and/or wheel are installed on Python 3.12+, by adding them to the get-pip.py invocation like so:

$ python get-pip.py setuptools wheel

Closes #200.

@edmorley
Copy link
Contributor Author

Please could I have review? :-)

@pradyunsg
Copy link
Member

Please could I have review? :-)

Yes, now that I have some bandwidth.

Currently `get-pip.py` installs not only pip, but also setuptools and wheel by default, unless the `--no-setuptools` / `--no-wheel` (or `PIP_NO_SETUPTOOLS` / `PIP_NO_WHEEL` env vars) are used. This has historically been necessary, however, modern versions of pip will now fallback to `pyproject.toml` (PEP 517: [1]) based builds (which will default to a setuptools backend, and thus automatically install setuptools and wheel in the isolated build environment) if either setuptools is not installed (as of pip 22.1: [2]), or if wheel is not installed (as of pip 23.1: [3]). In addition, as of Python 3.12, the stdlib's `ensurepip` and `venv` modules no longer install setuptools, and only install pip ([4]). As such, it is now time for `get-pip.py` to stop installing setuptools and wheel by default on Python 3.12+, in order to: - Act as another small step towards `pyproject.toml` / PEP 517 based builds eventually becoming the pip default. - Improve parity with the behaviour of `ensurepip` / `venv` on Python 3.12+. - Allow `get-pip.py` to focus on its primary responsibility: bootstrapping Pip. Closes pypa#200. [1]: https://peps.python.org/pep-0517/ [2]: pypa/pip#10717 [3]: pypa/pip#11871 [4]: python/cpython#101039
@edmorley edmorley force-pushed the no-setuptools-wheel branch from 2f463eb to def4aec Compare July 29, 2024 20:21
@edmorley
Copy link
Contributor Author

@pradyunsg Thank you for the review! I've made those changes. I wouldn't normally force push, but since this repo appears to exclusively use standard merges rather than squash and merge, I wanted to avoid a "Apply review feedback" type commit from ending up in the log on main. To see the diff between the last version and this, use:
https://github.com/pypa/get-pip/compare/2f463eb7e795d395a72f690eafca41b1b1fa8ea4..def4aec84b261b939137dd1c69eff0aabb4a7bf4

@edmorley edmorley requested a review from pradyunsg July 29, 2024 20:25
@edmorley
Copy link
Contributor Author

Testing with Python 3.12:

$ docker run --rm -it python:3.12 bash ... root@e4b186d6ffd7:/# cd tmp/ root@e4b186d6ffd7:/tmp# python -m venv .venv --without-pip root@e4b186d6ffd7:/tmp# source .venv/bin/activate (.venv) root@e4b186d6ffd7:/tmp# curl -fLO https://github.com/edmorley/get-pip/raw/no-setuptools-wheel/public/get-pip.py (.venv) root@e4b186d6ffd7:/tmp# python get-pip.py Collecting pip Downloading pip-24.2-py3-none-any.whl.metadata (3.6 kB) Downloading pip-24.2-py3-none-any.whl (1.8 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.8/1.8 MB 17.0 MB/s eta 0:00:00 Installing collected packages: pip Successfully installed pip-24.2 

And with Python 3.11:

$ docker run --rm -it python:3.11 bash ... root@6f0dbd95ed50:/# cd tmp/ root@6f0dbd95ed50:/tmp# python -m venv .venv --without-pip root@6f0dbd95ed50:/tmp# source .venv/bin/activate (.venv) root@6f0dbd95ed50:/tmp# curl -fLO https://github.com/edmorley/get-pip/raw/no-setuptools-wheel/public/get-pip.py (.venv) root@6f0dbd95ed50:/tmp# python get-pip.py Collecting pip Downloading pip-24.2-py3-none-any.whl.metadata (3.6 kB) Collecting setuptools Downloading setuptools-72.1.0-py3-none-any.whl.metadata (6.6 kB) Collecting wheel Downloading wheel-0.43.0-py3-none-any.whl.metadata (2.2 kB) Downloading pip-24.2-py3-none-any.whl (1.8 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.8/1.8 MB 57.8 MB/s eta 0:00:00 Downloading setuptools-72.1.0-py3-none-any.whl (2.3 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.3/2.3 MB 70.9 MB/s eta 0:00:00 Downloading wheel-0.43.0-py3-none-any.whl (65 kB) Installing collected packages: wheel, setuptools, pip Successfully installed pip-24.2 setuptools-72.1.0 wheel-0.43.0 
edmorley added a commit to heroku/buildpacks-python that referenced this pull request Aug 7, 2024
Currently the buildpack performs a system site-packages install of not only pip, but also setuptools and wheel. This has historically been necessary for pip to be able to build source distributions (sdists) for packages that don't ship with compatible wheels. However: - Thanks to PEP 518, packages can now (and many already do) specify an explicit build backend using `[build-system]` in their `pyproject.toml`. The dependencies specified in that config (such as setuptools and wheel) will be installed by pip into an isolated and ephemeral build environment as part of the source distribution build process. Such packages therefore don't need/use globally installed setuptools/wheel versions. - As of pip v22.1, pip will now default to the isolated build environment mode (along with a fallback legacy setuptools build backend), if the setuptools package isn't installed globally. This means that packages that haven't yet migrated to a PEP 518 `pyproject.toml` build backend config can still build even if setuptools isn't installed globally. There are a small number of rarely used packages in the wild that aren't compatible with build isolation mode, however, these typically require more build dependencies than just setuptools, which means they wouldn't have worked with this buildpack anyway. As such, it's no longer necessary for us to install setuptools and wheel globally. This matches the behaviour of the `venv` and `ensurepip` modules in Python 3.12+, where setuptools and wheel installation has also been removed. And it also matches the default behaviour of Poetry too, whose `install --sync` command removes any implicitly installed packages in the current environment (other than pip). See: https://peps.python.org/pep-0518/ https://pip.pypa.io/en/stable/reference/build-system/ https://pip.pypa.io/en/stable/reference/build-system/pyproject-toml/#build-isolation pypa/pip#10717 python/cpython#101039 pypa/get-pip#218 astral-sh/uv#2252 GUS-W-16437776.
edmorley added a commit to heroku/buildpacks-python that referenced this pull request Aug 7, 2024
Currently the buildpack performs a system site-packages install of not only pip, but also setuptools and wheel. This has historically been necessary for pip to be able to build source distributions (sdists) for packages that don't ship with compatible wheels. However: - Thanks to PEP 518, packages can now (and many already do) specify an explicit build backend using `[build-system]` in their `pyproject.toml`. The dependencies specified in that config (such as setuptools and wheel) will be installed by pip into an isolated and ephemeral build environment as part of the source distribution build process. Such packages therefore don't need/use globally installed setuptools/wheel versions. - As of pip v22.1, pip will now default to the isolated build environment mode (along with a fallback legacy setuptools build backend), if the setuptools package isn't installed globally. This means that packages that haven't yet migrated to a PEP 518 `pyproject.toml` build backend config can still build even if setuptools isn't installed globally. There are a small number of rarely used packages in the wild that aren't compatible with build isolation mode, however, these typically require more build dependencies than just setuptools, which means they wouldn't have worked with this buildpack anyway. As such, it's no longer necessary for us to install setuptools and wheel globally. This matches the behaviour of the `venv` and `ensurepip` modules in Python 3.12+, where setuptools and wheel installation has also been removed. And it also matches the default behaviour of Poetry too, whose `install --sync` command removes any implicitly installed packages in the current environment (other than pip). See: https://peps.python.org/pep-0518/ https://pip.pypa.io/en/stable/reference/build-system/ https://pip.pypa.io/en/stable/reference/build-system/pyproject-toml/#build-isolation pypa/pip#10717 python/cpython#101039 pypa/get-pip#218 astral-sh/uv#2252 GUS-W-16437776.
edmorley added a commit to heroku/buildpacks-python that referenced this pull request Aug 7, 2024
Currently the buildpack performs a system site-packages install of not only pip, but also setuptools and wheel. This has historically been necessary for pip to be able to build source distributions (sdists) for packages that don't ship with compatible wheels. However: - Thanks to PEP 518, packages can now (and many already do) specify an explicit build backend using `[build-system]` in their `pyproject.toml`. The dependencies specified in that config (such as setuptools and wheel) will be installed by pip into an isolated and ephemeral build environment as part of the source distribution build process. Such packages therefore don't need/use globally installed setuptools/wheel versions. - As of pip v22.1, pip will now default to the isolated build environment mode (along with a fallback legacy setuptools build backend), if the setuptools package isn't installed globally. This means that packages that haven't yet migrated to a PEP 518 `pyproject.toml` build backend config can still build even if setuptools isn't installed globally. There are a small number of rarely used packages in the wild that aren't compatible with build isolation mode, however, these typically require more build dependencies than just setuptools, which means they wouldn't have worked with this buildpack anyway. As such, it's no longer necessary for us to install setuptools and wheel globally. This matches the behaviour of the `venv` and `ensurepip` modules in Python 3.12+, where setuptools and wheel installation has also been removed. And it also matches the default behaviour of Poetry too, whose `install --sync` command removes any implicitly installed packages in the current environment (other than pip). See: https://peps.python.org/pep-0518/ https://pip.pypa.io/en/stable/reference/build-system/ https://pip.pypa.io/en/stable/reference/build-system/pyproject-toml/#build-isolation pypa/pip#10717 python/cpython#101039 pypa/get-pip#218 astral-sh/uv#2252 GUS-W-16437776.
@edmorley
Copy link
Contributor Author

Please could I have a re-review? :-)

@edmorley
Copy link
Contributor Author

Please may someone merge this, now that it's had review (and time for anyone else to comment/review/object)? :-)

@pradyunsg pradyunsg merged commit 3aeb098 into pypa:main Aug 26, 2024
2 checks passed
@pradyunsg
Copy link
Member

Thanks for the poke @edmorley! I've merged this but we can revert this if someone raises concerns around this.

@pradyunsg pradyunsg mentioned this pull request Aug 26, 2024
@edmorley edmorley deleted the no-setuptools-wheel branch August 26, 2024 18:03
@edmorley
Copy link
Contributor Author

Thank you :-)

As a reminder to anyone coming across this PR who wants to return to the old behaviour, you can update your get-pip.py usage to this to continue to install setuptools and wheel on Python 3.12+:

$ python get-pip.py setuptools wheel 
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
2 participants