Skip to content

Conversation

@serhiy-storchaka
Copy link
Member

@serhiy-storchaka serhiy-storchaka commented Apr 3, 2020

Patcher's __exit__() is now never called if its __enter__() is failed.
Returning true from __exit__() silences now the exception.

https://bugs.python.org/issue40126

Patcher's __exit__() is now never called if its __enter__() is failed. Returning true from __exit__() silences now the exception.
@serhiy-storchaka
Copy link
Member Author

Maybe ExitStack could be used here.

patcher.__exit__(*exc_info)
exit_stack = self._exit_stack
del self._exit_stack
return exit_stack.__exit__(*sys.exc_info())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be passing exc_info parameter passed to __exit__? Is there a case where exc_info can be different from sys.exc_info?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch! Thank you.

patcher.__exit__(*exc_info)
exit_stack = self._exit_stack
del self._exit_stack
return exit_stack.__exit__(*sys.exc_info())
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch! Thank you.

@serhiy-storchaka serhiy-storchaka merged commit 4b222c9 into python:master Apr 11, 2020
@miss-islington
Copy link
Contributor

Thanks @serhiy-storchaka for the PR 🌮🎉.. I'm working now to backport this PR to: 3.7, 3.8.
🐍🍒⛏🤖

@serhiy-storchaka serhiy-storchaka deleted the mock-patch-exit-stack branch April 11, 2020 07:59
@miss-islington
Copy link
Contributor

Sorry, @serhiy-storchaka, I could not cleanly backport this to 3.8 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker 4b222c9491d1700e9bdd98e6889b8d0ea1c7321e 3.8

@miss-islington
Copy link
Contributor

Sorry @serhiy-storchaka, I had trouble checking out the 3.7 backport branch.
Please backport using cherry_picker on command line.
cherry_picker 4b222c9491d1700e9bdd98e6889b8d0ea1c7321e 3.7

@tirkarthi
Copy link
Member

Thanks for the refactor. Can these cases be tested to catch them in future?

@cjw296
Copy link
Contributor

cjw296 commented Apr 11, 2020

@serhiy-storchaka - I have to be honest, bit disappointed to see changes like these going in without unit tests to verify their behaviour. Would you be able to put in another PR to add these tests?

@serhiy-storchaka
Copy link
Member Author

I agree with you, but it is very hard to write any realistic tests for subtle behavior changes caused by this PR. I'll try more, but I don't have any tests yet.

At least the code looks now clearer and the existing tests are passed.

serhiy-storchaka added a commit to serhiy-storchaka/cpython that referenced this pull request Apr 12, 2020
…thonGH-19351) Patcher's __exit__() is now never called if its __enter__() is failed. Returning true from __exit__() silences now the exception.. (cherry picked from commit 4b222c9) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
@bedevere-bot
Copy link

GH-19483 is a backport of this pull request to the 3.8 branch.

@bedevere-bot
Copy link

GH-19484 is a backport of this pull request to the 3.7 branch.

serhiy-storchaka added a commit to serhiy-storchaka/cpython that referenced this pull request Apr 12, 2020
…thonGH-19351) Patcher's __exit__() is now never called if its __enter__() is failed. Returning true from __exit__() silences now the exception.. (cherry picked from commit 4b222c9) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
serhiy-storchaka added a commit that referenced this pull request Apr 12, 2020
…-19351) (GH-19483) Patcher's __exit__() is now never called if its __enter__() is failed. Returning true from __exit__() silences now the exception. (cherry picked from commit 4b222c9) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
serhiy-storchaka added a commit that referenced this pull request Apr 12, 2020
…-19351) (GH-19484) Patcher's __exit__() is now never called if its __enter__() is failed. Returning true from __exit__() silences now the exception. (cherry picked from commit 4b222c9) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 Windows7 SP1 3.8 has failed when building commit ee249d7.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/163/builds/250) and take a look at the build logs.
  4. Check if the failure is related to this commit (ee249d7) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/163/builds/250

Failed tests:

  • test_asyncio

Failed subtests:

  • test_huge_content - test.test_asyncio.test_sock_lowlevel.SelectEventLoopTests
  • test_huge_content - test.test_asyncio.test_sock_lowlevel.ProactorEventLoopTests

Summary of the results of the build (if available):

== Tests result: FAILURE then FAILURE ==

389 tests OK.

10 slowest tests:

  • test_multiprocessing_spawn: 3 min 58 sec
  • test_tokenize: 2 min 23 sec
  • test_venv: 2 min 4 sec
  • test_tools: 2 min 1 sec
  • test_concurrent_futures: 1 min 51 sec
  • test_io: 1 min 45 sec
  • test_mmap: 1 min 42 sec
  • test_lib2to3: 1 min 39 sec
  • test_asyncio: 1 min 17 sec
  • test_largefile: 1 min 8 sec

1 test failed:
test_asyncio

33 tests skipped:
test_curses test_dbm_gnu test_dbm_ndbm test_devpoll test_epoll
test_fcntl test_fork1 test_gdb test_grp test_ioctl test_kqueue
test_multiprocessing_fork test_multiprocessing_forkserver test_nis
test_openpty test_ossaudiodev test_pipes test_poll test_posix
test_pty test_pwd test_readline test_resource test_spwd
test_syslog test_threadsignals test_tix test_tk test_ttk_guionly
test_wait3 test_wait4 test_xxtestfuzz test_zipfile64

1 re-run test:
test_asyncio

Total duration: 14 min 24 sec

Click to see traceback logs
Traceback (most recent call last): File "C:\buildbot.python.org\3.8.kloth-win64\build\lib\test\test_asyncio\test_sock_lowlevel.py", line 170, in test_huge_content self.loop.run_until_complete( File "C:\buildbot.python.org\3.8.kloth-win64\build\lib\asyncio\base_events.py", line 616, in run_until_complete return future.result() File "C:\buildbot.python.org\3.8.kloth-win64\build\lib\test\test_asyncio\test_sock_lowlevel.py", line 157, in _basetest_huge_content data = await self.loop.sock_recv(sock, DATA_SIZE) File "C:\buildbot.python.org\3.8.kloth-win64\build\lib\asyncio\selector_events.py", line 368, in sock_recv return await fut File "C:\buildbot.python.org\3.8.kloth-win64\build\lib\asyncio\selector_events.py", line 379, in _sock_recv data = sock.recv(n) ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host Traceback (most recent call last): File "C:\buildbot.python.org\3.8.kloth-win64\build\lib\asyncio\windows_events.py", line 453, in finish_recv return ov.getresult() OSError: [WinError 64] The specified network name is no longer available Traceback (most recent call last): File "C:\buildbot.python.org\3.8.kloth-win64\build\lib\test\test_asyncio\test_sock_lowlevel.py", line 170, in test_huge_content self.loop.run_until_complete( File "C:\buildbot.python.org\3.8.kloth-win64\build\lib\asyncio\base_events.py", line 616, in run_until_complete return future.result() File "C:\buildbot.python.org\3.8.kloth-win64\build\lib\test\test_asyncio\test_sock_lowlevel.py", line 157, in _basetest_huge_content data = await self.loop.sock_recv(sock, DATA_SIZE) File "C:\buildbot.python.org\3.8.kloth-win64\build\lib\asyncio\proactor_events.py", line 693, in sock_recv return await self._proactor.recv(sock, n) File "C:\buildbot.python.org\3.8.kloth-win64\build\lib\asyncio\windows_events.py", line 808, in _poll value = callback(transferred, key, ov) File "C:\buildbot.python.org\3.8.kloth-win64\build\lib\asyncio\windows_events.py", line 457, in finish_recv raise ConnectionResetError(*exc.args) ConnectionResetError: [WinError 64] The specified network name is no longer available 
@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 FreeBSD Shared 3.8 has failed when building commit ee249d7.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/18/builds/196) and take a look at the build logs.
  4. Check if the failure is related to this commit (ee249d7) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/18/builds/196

Failed tests:

  • test_multiprocessing_fork

Summary of the results of the build (if available):

==

Click to see traceback logs
remote: Enumerating objects: 9, done.  remote: Counting objects: 11% (1/9)  remote: Counting objects: 22% (2/9)  remote: Counting objects: 33% (3/9)  remote: Counting objects: 44% (4/9)  remote: Counting objects: 55% (5/9)  remote: Counting objects: 66% (6/9)  remote: Counting objects: 77% (7/9)  remote: Counting objects: 88% (8/9)  remote: Counting objects: 100% (9/9)  remote: Counting objects: 100% (9/9), done.  remote: Total 10 (delta 8), reused 8 (delta 8), pack-reused 1  From https://github.com/python/cpython * branch 3.8 -> FETCH_HEAD Reset branch '3.8' test_msilib skipped -- No module named '_msi' test_tix skipped -- No module named '_tkinter' test_startfile skipped -- object <module 'os' from '/usr/home/buildbot/python/3.8.koobs-freebsd-564d/build/Lib/os.py'> has no attribute 'startfile' test_winsound skipped -- No module named 'winsound' test_dbm_gnu skipped -- No module named '_gdbm' /usr/home/buildbot/python/3.8.koobs-freebsd-564d/build/Lib/multiprocessing/resource_tracker.py:216: UserWarning: resource_tracker: There appear to be 1 leaked shared_memory objects to clean up at shutdown warnings.warn('resource_tracker: There appear to be %d ' /usr/home/buildbot/python/3.8.koobs-freebsd-564d/build/Lib/multiprocessing/resource_tracker.py:229: UserWarning: resource_tracker: '/psm_96976737': [Errno 2] No such file or directory: '/psm_96976737' warnings.warn('resource_tracker: %r: %s' % (name, e)) stty: stdin isn't a terminal test_flock (__main__.FNTLEINTRTest) ... ok test_lockf (__main__.FNTLEINTRTest) ... ok test_read (__main__.OSEINTRTest) ... ok test_wait (__main__.OSEINTRTest) ... ok test_wait3 (__main__.OSEINTRTest) ... ok test_wait4 (__main__.OSEINTRTest) ... ok test_waitpid (__main__.OSEINTRTest) ... ok test_write (__main__.OSEINTRTest) ... ok test_devpoll (__main__.SelectEINTRTest) ... skipped 'need select.devpoll' test_epoll (__main__.SelectEINTRTest) ... skipped 'need select.epoll' test_kqueue (__main__.SelectEINTRTest) ... ok test_poll (__main__.SelectEINTRTest) ... ok test_select (__main__.SelectEINTRTest) ... ok test_sigtimedwait (__main__.SignalEINTRTest) ... ok test_sigwaitinfo (__main__.SignalEINTRTest) ... ok test_accept (__main__.SocketEINTRTest) ... ok test_open (__main__.SocketEINTRTest) ... ok test_os_open (__main__.SocketEINTRTest) ... ok test_recv (__main__.SocketEINTRTest) ... ok test_recvmsg (__main__.SocketEINTRTest) ... ok test_send (__main__.SocketEINTRTest) ... ok test_sendall (__main__.SocketEINTRTest) ... ok test_sendmsg (__main__.SocketEINTRTest) ... ok test_sleep (__main__.TimeEINTRTest) ... ok ---------------------------------------------------------------------- Ran 24 tests in 12.145s OK (skipped=2) test_epoll skipped -- test works only on Linux 2.6 test_winreg skipped -- No module named 'winreg' /usr/home/buildbot/python/3.8.koobs-freebsd-564d/build/Lib/multiprocessing/resource_tracker.py:216: UserWarning: resource_tracker: There appear to be 1 leaked shared_memory objects to clean up at shutdown warnings.warn('resource_tracker: There appear to be %d ' /usr/home/buildbot/python/3.8.koobs-freebsd-564d/build/Lib/multiprocessing/resource_tracker.py:229: UserWarning: resource_tracker: '/psm_bab719c3': [Errno 2] No such file or directory: '/psm_bab719c3' warnings.warn('resource_tracker: %r: %s' % (name, e)) test_ttk_guionly skipped -- No module named '_tkinter' test_ttk_textonly skipped -- No module named '_tkinter' test_spwd skipped -- No module named 'spwd' test_idle skipped -- No module named '_tkinter' test_ossaudiodev skipped -- [Errno 2] No such file or directory: '/dev/dsp' test_devpoll skipped -- test works only on Solaris OS family test_ioctl skipped -- Unable to open /dev/tty test_tcl skipped -- No module named '_tkinter' test_winconsoleio skipped -- test only relevant on win32 test_tk skipped -- No module named '_tkinter' test_zipfile64 skipped -- test requires loads of disk-space bytes and a long time to run test_turtle skipped -- No module named '_tkinter' Kill <TestWorkerProcess #2 running test=test_multiprocessing_fork pid=78047 time=37 min 30 sec> Timeout (0:25:00)! Thread 0x00000008012fff00 (most recent call first): File "/usr/home/buildbot/python/3.8.koobs-freebsd-564d/build/Lib/multiprocessing/connection.py", line 379 in _recv File "/usr/home/buildbot/python/3.8.koobs-freebsd-564d/build/Lib/multiprocessing/connection.py", line 414 in _recv_bytes File "/usr/home/buildbot/python/3.8.koobs-freebsd-564d/build/Lib/multiprocessing/connection.py", line 250 in recv File "/usr/home/buildbot/python/3.8.koobs-freebsd-564d/build/Lib/multiprocessing/pool.py", line 576 in _handle_results File "/usr/home/buildbot/python/3.8.koobs-freebsd-564d/build/Lib/threading.py", line 870 in run File "/usr/home/buildbot/python/3.8.koobs-freebsd-564d/build/Lib/threading.py", line 932 in _bootstrap_inner File "/usr/home/buildbot/python/3.8.koobs-freebsd-564d/build/Lib/threading.py", line 890 in _bootstrap Thread 0x0000000801302c00 (most recent call first): File "/usr/home/buildbot/python/3.8.koobs-freebsd-564d/build/Lib/multiprocessing/pool.py", line 528 in _handle_tasks File "/usr/home/buildbot/python/3.8.koobs-freebsd-564d/build/Lib/threading.py", line 870 in run File "/usr/home/buildbot/python/3.8.koobs-freebsd-564d/build/Lib/threading.py", line 932 in _bootstrap_inner File "/usr/home/buildbot/python/3.8.koobs-freebsd-564d/build/Lib/threading.py", line 890 in _bootstrap Thread 0x0000000801300e00 (most recent call first): File "/usr/home/buildbot/python/3.8.koobs-freebsd-564d/build/Lib/selectors.py", line 415 in select File "/usr/home/buildbot/python/3.8.koobs-freebsd-564d/build/Lib/multiprocessing/connection.py", line 931 in wait File "/usr/home/buildbot/python/3.8.koobs-freebsd-564d/build/Lib/multiprocessing/pool.py", line 499 in _wait_for_updates File "/usr/home/buildbot/python/3.8.koobs-freebsd-564d/build/Lib/multiprocessing/pool.py", line 519 in _handle_workers File "/usr/home/buildbot/python/3.8.koobs-freebsd-564d/build/Lib/threading.py", line 870 in run File "/usr/home/buildbot/python/3.8.koobs-freebsd-564d/build/Lib/threading.py", line 932 in _bootstrap_inner File "/usr/home/buildbot/python/3.8.koobs-freebsd-564d/build/Lib/threading.py", line 890 in _bootstrap Thread 0x0000000800ac1000 (most recent call first): File "/usr/home/buildbot/python/3.8.koobs-freebsd-564d/build/Lib/multiprocessing/popen_fork.py", line 27 in poll File "/usr/home/buildbot/python/3.8.koobs-freebsd-564d/build/Lib/multiprocessing/popen_fork.py", line 47 in wait File "/usr/home/buildbot/python/3.8.koobs-freebsd-564d/build/Lib/multiprocessing/process.py", line 149 in join File "/usr/home/buildbot/python/3.8.koobs-freebsd-564d/build/Lib/multiprocessing/pool.py", line 729 in _terminate_pool File "/usr/home/buildbot/python/3.8.koobs-freebsd-564d/build/Lib/multiprocessing/util.py", line 224 in __call__ File "/usr/home/buildbot/python/3.8.koobs-freebsd-564d/build/Lib/multiprocessing/pool.py", line 654 in terminate File "/usr/home/buildbot/python/3.8.koobs-freebsd-564d/build/Lib/multiprocessing/pool.py", line 736 in __exit__ File "/usr/home/buildbot/python/3.8.koobs-freebsd-564d/build/Lib/test/_test_multiprocessing.py", line 2664 in test_enter File "/usr/home/buildbot/python/3.8.koobs-freebsd-564d/build/Lib/unittest/case.py", line 633 in _callTestMethod File "/usr/home/buildbot/python/3.8.koobs-freebsd-564d/build/Lib/unittest/case.py", line 676 in run File "/usr/home/buildbot/python/3.8.koobs-freebsd-564d/build/Lib/unittest/case.py", line 736 in __call__ File "/usr/home/buildbot/python/3.8.koobs-freebsd-564d/build/Lib/unittest/suite.py", line 122 in run File "/usr/home/buildbot/python/3.8.koobs-freebsd-564d/build/Lib/unittest/suite.py", line 84 in __call__ File "/usr/home/buildbot/python/3.8.koobs-freebsd-564d/build/Lib/unittest/suite.py", line 122 in run File "/usr/home/buildbot/python/3.8.koobs-freebsd-564d/build/Lib/unittest/suite.py", line 84 in __call__ File "/usr/home/buildbot/python/3.8.koobs-freebsd-564d/build/Lib/unittest/suite.py", line 122 in run File "/usr/home/buildbot/python/3.8.koobs-freebsd-564d/build/Lib/unittest/suite.py", line 84 in __call__ File "/usr/home/buildbot/python/3.8.koobs-freebsd-564d/build/Lib/unittest/runner.py", line 176 in run File "/usr/home/buildbot/python/3.8.koobs-freebsd-564d/build/Lib/test/support/__init__.py", line 2030 in _run_suite File "/usr/home/buildbot/python/3.8.koobs-freebsd-564d/build/Lib/test/support/__init__.py", line 2126 in run_unittest File "/usr/home/buildbot/python/3.8.koobs-freebsd-564d/build/Lib/test/libregrtest/runtest.py", line 209 in _test_module File "/usr/home/buildbot/python/3.8.koobs-freebsd-564d/build/Lib/test/libregrtest/runtest.py", line 234 in _runtest_inner2 File "/usr/home/buildbot/python/3.8.koobs-freebsd-564d/build/Lib/test/libregrtest/runtest.py", line 270 in _runtest_inner File "/usr/home/buildbot/python/3.8.koobs-freebsd-564d/build/Lib/test/libregrtest/runtest.py", line 153 in _runtest File "/usr/home/buildbot/python/3.8.koobs-freebsd-564d/build/Lib/test/libregrtest/runtest.py", line 193 in runtest File "/usr/home/buildbot/python/3.8.koobs-freebsd-564d/build/Lib/test/libregrtest/main.py", line 318 in rerun_failed_tests File "/usr/home/buildbot/python/3.8.koobs-freebsd-564d/build/Lib/test/libregrtest/main.py", line 691 in _main File "/usr/home/buildbot/python/3.8.koobs-freebsd-564d/build/Lib/test/libregrtest/main.py", line 634 in main File "/usr/home/buildbot/python/3.8.koobs-freebsd-564d/build/Lib/test/libregrtest/main.py", line 712 in main File "/usr/home/buildbot/python/3.8.koobs-freebsd-564d/build/Lib/test/__main__.py", line 2 in <module> File "/usr/home/buildbot/python/3.8.koobs-freebsd-564d/build/Lib/runpy.py", line 87 in _run_code File "/usr/home/buildbot/python/3.8.koobs-freebsd-564d/build/Lib/runpy.py", line 194 in _run_module_as_main Cannot open file '/usr/home/buildbot/python/3.8.koobs-freebsd-564d/build/test-results.xml' for upload
@cjw296
Copy link
Contributor

cjw296 commented Apr 14, 2020

@serhiy-storchaka - what is the case that is difficult to test? I've had a quick scan of the issue, and neither the original bug report, around something attempting to clear patchings (just have the unit test to the same thing), nor your observation that __exit__ should not be called if __enter__ fails (worst case: mock both, first to raise, second to record and check your assertions, although likely better to do something that will always cause an error in __enter__ and check __exit__ isn't called by way of it not doing the wrong thing) seem unusually difficult to test, what am I missing?

@serhiy-storchaka serhiy-storchaka removed their assignment Dec 29, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type-bug An unexpected behavior, bug, or error

6 participants