Closed
Description
Bug report
Bug description:
multiprocessing.Pool with spawn method breaks logging.handlers.QueueHandler configuration
import logging import logging.config import multiprocessing import time def _init(q): logging.config.dictConfig({ 'version': 1, 'disable_existing_loggers': True, 'handlers': {'log_to_parent': {'class': 'logging.handlers.QueueHandler', 'queue': q}}, 'root': {'handlers': ['log_to_parent'], 'level': 'DEBUG'} }) logging.getLogger().info('log from child') if __name__ == '__main__': multiprocessing.set_start_method('spawn') with multiprocessing.Manager() as manager: # q = manager.Queue() q = multiprocessing.Queue() # listen for log messages from child processes # ... with multiprocessing.Pool(processes=1, maxtasksperchild=1, initializer=_init, initargs=(q,)) as pool: time.sleep(1)got exception (when q = manager.Queue() or q = multiprocessing.Queue())
Process SpawnPoolWorker-2: Traceback (most recent call last): File "/home/foo/miniconda3/envs/py312forge/lib/python3.12/logging/config.py", line 581, in configure handler = self.configure_handler(handlers[name]) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/foo/miniconda3/envs/py312forge/lib/python3.12/logging/config.py", line 792, in configure_handler proxy_queue = MM().Queue() ^^^^ File "/home/foo/miniconda3/envs/py312forge/lib/python3.12/multiprocessing/context.py", line 57, in Manager m.start() File "/home/foo/miniconda3/envs/py312forge/lib/python3.12/multiprocessing/managers.py", line 562, in start self._process.start() File "/home/foo/miniconda3/envs/py312forge/lib/python3.12/multiprocessing/process.py", line 118, in start assert not _current_process._config.get('daemon'), \ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ AssertionError: daemonic processes are not allowed to have children The above exception was the direct cause of the following exception: ... CPython versions tested on:
3.12
Operating systems tested on:
Linux
Linked PRs
- gh-121723: Relax constraints on queue objects for
logging.handlers.QueueHandler. #122154 - [3.13] gh-121723: Relax constraints on queue objects for
logging.handlers.QueueHandler. (GH-122154) #122603 - [3.12] gh-121723: Relax constraints on queue objects for
logging.handlers.QueueHandler. (GH-122154) #122604 - GH-121723: Skip
test_config_queue_handler_multiprocessing_contextin emulated JIT CI #122969 - [3.13] GH-121723: Skip test_config_queue_handler_multiprocessing_context in emulated JIT CI (GH-122969) #122991
Metadata
Metadata
Assignees
Projects
Status
Done