Issue #18999: Make multiprocessing use context objects.

This allows different parts of a program to use different methods for starting processes without interfering with each other.

Author: shibturn

Doc/library/multiprocessing.rst

@@ -98,8 +98,8 @@ necessary, see :ref:`multiprocessing-programming`.
 
 
 
-Start methods
-~~~~~~~~~~~~~
+Contexts and start methods
+~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 Depending on the platform, :mod:`multiprocessing` supports three ways
 to start a process. These *start methods* are
@@ -132,7 +132,7 @@ to start a process. These *start methods* are
  unnecessary resources are inherited.
 
  Available on Unix platforms which support passing file descriptors
- over unix pipes.
+ over Unix pipes.
 
 Before Python 3.4 *fork* was the only option available on Unix. Also,
 prior to Python 3.4, child processes would inherit all the parents
@@ -153,18 +153,46 @@ example::
 
  import multiprocessing as mp
 
- def foo():
- print('hello')
+ def foo(q):
+ q.put('hello')
 
  if __name__ == '__main__':
  mp.set_start_method('spawn')
- p = mp.Process(target=foo)
+ q = mp.Queue()
+ p = mp.Process(target=foo, args=(q,))
  p.start()
+ print(q.get())
  p.join()
 
 :func:`set_start_method` should not be used more than once in the
 program.
 
+Alternatively, you can use :func:`get_context` to obtain a context
+object. Context objects have the same API as the multiprocessing
+module, and allow one to use multiple start methods in the same
+program. ::
+
+ import multiprocessing as mp
+
+ def foo(q):
+ q.put('hello')
+
+ if __name__ == '__main__':
+ ctx = mp.get_context('spawn')
+ q = ctx.Queue()
+ p = ctx.Process(target=foo, args=(q,))
+ p.start()
+ print(q.get())
+ p.join()
+
+Note that objects related to one context may not be compatible with
+processes for a different context. In particular, locks created using
+the *fork* context cannot be passed to a processes started using the
+*spawn* or *forkserver* start methods.
+
+A library which wants to use a particular start method should probably
+use :func:`get_context` to avoid interfering with the choice of the
+library user.
 
 
 Exchanging objects between processes
@@ -859,11 +887,30 @@ Miscellaneous
 
  .. versionadded:: 3.4
 
-.. function:: get_start_method()
+.. function:: get_context(method=None)
+
+ Return a context object which has the same attributes as the
+ :mod:`multiprocessing` module.
 
- Return the current start method. This can be ``'fork'``,
- ``'spawn'`` or ``'forkserver'``. ``'fork'`` is the default on
- Unix, while ``'spawn'`` is the default on Windows.
+ If *method* is *None* then the default context is returned.
+ Otherwise *method* should be ``'fork'``, ``'spawn'``,
+ ``'forkserver'``. :exc:`ValueError` is raised if the specified
+ start method is not available.
+
+ .. versionadded:: 3.4
+
+.. function:: get_start_method(allow_none=False)
+
+ Return the name of start method used for starting processes.
+
+ If the start method has not been fixed and *allow_none* is false,
+ then the start method is fixed to the default and the name is
+ returned. If the start method has not been fixed and *allow_none*
+ is true then *None* is returned.
+
+ The return value can be ``'fork'``, ``'spawn'``, ``'forkserver'``
+ or *None*. ``'fork'`` is the default on Unix, while ``'spawn'`` is
+ the default on Windows.
 
  .. versionadded:: 3.4
 
@@ -1785,7 +1832,7 @@ Process Pools
 One can create a pool of processes which will carry out tasks submitted to it
 with the :class:`Pool` class.
 
-.. class:: Pool([processes[, initializer[, initargs[, maxtasksperchild]]]])
+.. class:: Pool([processes[, initializer[, initargs[, maxtasksperchild [, context]]]]])
 
  A process pool object which controls a pool of worker processes to which jobs
  can be submitted. It supports asynchronous results with timeouts and
@@ -1805,6 +1852,13 @@ with the :class:`Pool` class.
  unused resources to be freed. The default *maxtasksperchild* is None, which
  means worker processes will live as long as the pool.
 
+ .. versionadded:: 3.4
+ *context* can be used to specify the context used for starting
+ the worker processes. Usually a pool is created using the
+ function :func:`multiprocessing.Pool` or the :meth:`Pool` method
+ of a context object. In both cases *context* is set
+ appropriately.
+
  .. note::
 
  Worker processes within a :class:`Pool` typically live for the complete