This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Created on 2014-06-20 22:13 by cool-RR, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
issue21817.patch Claudiu.Popa, 2014-06-22 05:47 review
issue21817_1.patch Claudiu.Popa, 2015-01-17 16:15
issue21817_2.patch Claudiu.Popa, 2015-01-17 18:05
Messages (9)
msg221128 - (view) Author: Ram Rachum (cool-RR) * Date: 2014-06-20 22:13
When you use `concurrent.futures.ProcessPoolExecutor` and an exception is raised in the created process, it doesn't show you the traceback. This makes debugging very annoying. Example file: #!python import sys import concurrent.futures def f(): # Successful assert: assert True return g() def g(): # Hard-to-find failing assert: assert False if __name__ == '__main__': with concurrent.futures.ProcessPoolExecutor(max_workers=1) as executor: assert isinstance(executor, concurrent.futures.Executor) future = executor.submit(f) future.result() print('Main process finished') If you run this under Windows, you get this: Traceback (most recent call last): File "./bug.py", line 20, in <module> future.result() File "c:\python34\lib\concurrent\futures\_base.py", line 402, in result return self.__get_result() File "c:\python34\lib\concurrent\futures\_base.py", line 354, in __get_result raise self._exception AssertionError This is the traceback of the main process, while we want the traceback of the process that failed.
msg221218 - (view) Author: PCManticore (Claudiu.Popa) * (Python triager) Date: 2014-06-22 05:47
Hello. Here's a patch based on c4f92b597074, which adds something similar to multiprocessing.pool. The output after the patch is: concurrent.futures.process.RemoteTraceback: """ Traceback (most recent call last): File "D:\Projects\cpython\lib\concurrent\futures\process.py", line 153, in _process_worker r = call_item.fn(*call_item.args, **call_item.kwargs) File "D:\Projects\cpython\PCbuild\a.py", line 9, in f return g() File "D:\Projects\cpython\PCbuild\a.py", line 13, in g assert False AssertionError """ The above exception was the direct cause of the following exception: Traceback (most recent call last): File "a.py", line 20, in <module> future.result() File "D:\Projects\cpython\lib\concurrent\futures\_base.py", line 402, in result return self.__get_result() File "D:\Projects\cpython\lib\concurrent\futures\_base.py", line 354, in __get_result raise self._exception AssertionError It's a little better than the current output, even though it's a little verbose.
msg225649 - (view) Author: PCManticore (Claudiu.Popa) * (Python triager) Date: 2014-08-22 07:36
Any type of feedback regarding this approach will be appreciated.
msg225660 - (view) Author: Ram Rachum (cool-RR) * Date: 2014-08-22 11:43
Hi Claudiu, sorry for the silence. This output looks great. I'd love to see that go into Python.
msg234174 - (view) Author: PCManticore (Claudiu.Popa) * (Python triager) Date: 2015-01-17 16:15
Here's the updated version of this patch.
msg234181 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2015-01-17 17:54
Hi, here are some comments about the patch: - RemoteTraceback, ExceptionWithTraceback, rebuild_exc should be private (i.e. with a leading underscore) - in test_traceback, you can use the context manager form of assertRaises instead of the try..except..else construct
msg234186 - (view) Author: PCManticore (Claudiu.Popa) * (Python triager) Date: 2015-01-17 18:05
Thanks, Antoine! Here's the new version, with your comments addressed.
msg234191 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2015-01-17 19:02
New changeset a36b402b099b by Antoine Pitrou in branch 'default': Issue #21817: When an exception is raised in a task submitted to a ProcessPoolExecutor, the remote traceback is now displayed in the parent process. https://hg.python.org/cpython/rev/a36b402b099b
msg234192 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2015-01-17 19:03
Thank you very much. Everything is now committed.
History
Date User Action Args
2022-04-11 14:58:05adminsetgithub: 66016
2015-01-17 19:03:59pitrousetstatus: open -> closed
resolution: fixed
messages: + msg234192

stage: patch review -> resolved
2015-01-17 19:02:48python-devsetnosy: + python-dev
messages: + msg234191
2015-01-17 18:05:08Claudiu.Popasetfiles: + issue21817_2.patch

messages: + msg234186
2015-01-17 17:54:35pitrousetnosy: + pitrou
messages: + msg234181
2015-01-17 16:15:32Claudiu.Popasetfiles: + issue21817_1.patch
type: behavior -> enhancement
messages: + msg234174

versions: - Python 3.4
2014-08-22 11:43:48cool-RRsetmessages: + msg225660
2014-08-22 07:36:34Claudiu.Popasetmessages: + msg225649
2014-06-22 05:47:40Claudiu.Popasetfiles: + issue21817.patch

versions: + Python 3.5
keywords: + patch
nosy: + Claudiu.Popa

messages: + msg221218
stage: patch review
2014-06-21 19:58:25ned.deilysetnosy: + bquinlan
2014-06-21 01:06:15josh.rsetnosy: + josh.r
2014-06-20 22:13:22cool-RRcreate