Skip to content

Conversation

@nascheme
Copy link
Member

@nascheme nascheme commented Dec 24, 2024

Edit: this PR has been revised to make the behavior dependent on the build of Python used. The free-threaded build defaults to inheriting the context of the caller of Thread.start(). The default behavior can be overridden by an -X var or by an environment variable. In the long term I think the behavior should be consistent between builds, both inheriting by default. The flag gives users of the default build an opt-in setting.

  • Add the sys.flags.thread_inherit_context flag.

  • This flag is set to true by default on the free-threaded build and false otherwise. If the flag is true, starting a new thread using threading.Thread will, by default, use a copy of the contextvars.Context from the caller of threading.Thread.start rather than using an empty context.

  • Add the -X thread_inherit_context command-line option and PYTHON_THREAD_INHERIT_CONTEXT environment variable, which set the sys.flags.thread_inherit_context flag.

  • Add the context keyword parameter to threading.Thread. It can be used to explicitly pass a context value to be used by a new thread.

  • Make the _contextvars module built-in.

This was motivated by making the warnings module work more reliably when threads and asyncio are used in combination with catch_warnings. I'm working on a PR that will make warnings use a contextvar for the filtering state.

Making new threads inherit context is not a new idea, it was suggested around the 3.10 release timeframe, gh-86981.


📚 Documentation preview 📚: https://cpython-previews--128209.org.readthedocs.build/

@nascheme
Copy link
Member Author

nascheme commented Jan 3, 2025

I wonder if threading.Thread is the correct layer to do this at. Perhaps it should be done at a lower level, e.g. inside Modules/_threadmodule.c.

Use the *target* name if *name* argument is omitted.

.. versionchanged:: 3.14
Added the *context* parameter. Previously threads always ran with an empty
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I'd make this more direct about the behavior change, perhaps something like:

"Threads now inherit the context of the caller of Thread.start instead of starting with an empty context. The context parameter was added. Pass a new contextvars.Context() if your thread requires an empty context."

This default change is a behavior change. Expect that to trip someones existing code up. I don't have the context (pun intended) as to how disruptive that could be. Per the PEP-387 breaking change policy we'd want to wait at least two releases. Which isn't so satisfying given the reasons you want this.

But a compromise could be considered (unsure if this is really wise) if needed: Change the default sooner than the deprecation when running in an experimental free-threading cpython build?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I asked Sam what he thought about making this behavior enabled by default on free-threaded builds and he's okay with the idea. I've updated this PR to take that approach. I think we can wait a couple of releases to decide what to do about the default build. If it seems there is little breakage, we can change the default there too after a suitable period.

@nascheme nascheme changed the title Add 'context' keyword parameter to Thread. gh-128555: Add 'context' keyword parameter to Thread. Jan 6, 2025
* Add ``sys.flags.inherit_context``. * Add ``-X inherit_context`` and :envvar:`PYTHON_INHERIT_CONTEXT`
@nascheme nascheme force-pushed the thread_inherit_context branch from 19d637a to 6d00c2a Compare February 6, 2025 22:29
@nascheme
Copy link
Member Author

nascheme commented Apr 9, 2025

Merged along with gh-130010.

@nascheme nascheme closed this Apr 9, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

3 participants