Bug #13632
closed
Not processable interrupt queue for a thread after it's notified that FD is closed in some other thread.
Description
In the bugfix for https://bugs.ruby-lang.org/issues/13076 has been introduced another bug, caused by a busy waiting in rb_notify_fd_close method, while the FD is being released. During this waiting, it pumps huge amounts of the ruby_error_stream_closed errors into thread's interrupt queue, which almost all stay there unprocessed. It can be up for several hundred of exceptions in the queue, depending on circumstances.
a = [] t = [] 10.times do r,w = IO.pipe a << [r,w] t << Thread.new do while r.gets end rescue IOError # Interrupt queue is full and all IO is broken Thread.current.pending_interrupt? # Expected to be false, because it's already rescued IO.sysopen ("/dev/tty") # Expected not to throw an error. On each such call, it dequeues the next item from interrupt queue until there's none end end a.each do |r,w| w.puts "test" w.close r.close end t.each do |th| th.join end Output:
Traceback (most recent call last): 1: from test2.rb:9:in `block (2 levels) in <main>' test2.rb:9:in `sysopen': stream closed in another thread (IOError) 
Updated by nvashchenko (Nikolay Vashchenko) over 8 years ago
- Description updated (diff)
Updated by nvashchenko (Nikolay Vashchenko) over 8 years ago
- Description updated (diff)
Updated by normalperson (Eric Wong) over 8 years ago
sir.nickolas@gmail.com wrote:
Bug #13632: Not processable interrupt queue for a thread after it's notified that FD is closed in some other thread.
https://bugs.ruby-lang.org/issues/13632
Investigating. I've been looking at IO close notification for
[Feature #13618] (auto-Fiber) anyways.
Updated by Anonymous over 8 years ago
- Status changed from Open to Closed
Applied in changeset trunk|r59020.
IO#close: do not enqueue redundant interrupts
Enqueuing multiple errors for one event causes spurious errors
down the line, as reported by Nikolay Vashchenko in
https://bugs.ruby-lang.org/issues/13632
- thread.c (rb_notify_fd_close): do not enqueue multiple interrupts
[ruby-core:81581] [Bug #13632] - test/ruby/test_io.rb (test_single_exception_on_close):
new test based on script from Nikolay
Updated by normalperson (Eric Wong) over 8 years ago
sir.nickolas@gmail.com wrote:
r59020 should fix it trivially in trunk.
Backporting to <= 2.4 is only a little different due to the
data structure change:
s/wfd->fd = -1/th->waiting_fd = -1/ https://80x24.org/spew/20170606001646.22889-1-e@80x24.org/raw Thanks for the report!
Updated by ko1 (Koichi Sasada) over 8 years ago
- Description updated (diff)
Updated by normalperson (Eric Wong) over 8 years ago
Eric Wong normalperson@yhbt.net wrote:
sir.nickolas@gmail.com wrote:
r59020 should fix it trivially in trunk.
Make that r59028 :x r59020 interacted badly with r57422
Backporting to <= 2.4 is only a little different due to the
data structure change:
r59028 backporting is more difficult.
Below are links to backported patches from trunk to fix [Bug #13632] for
Ruby 2.4 maintenance branches and earlier. I mainly wanted to
backport r59028, but that depends on r58812 (originally intended
as a pure performance optimization).
I'd rather not change r59028 to something unrecognizable from
what is in trunk for backporting, as that might negatively
impact future backports.
r58812: speed up IO#close with many threads
https://80x24.org/spew/20170607195901.18958-2-e@80x24.org/raw
r59028: IO#close: do not enqueue redundant interrupts (take #2)
https://80x24.org/spew/20170607195901.18958-3-e@80x24.org/raw
test/ruby/test_io.rb | 22 ++++++++++++++++++++++
thread.c | 33 ++++++++++++++++++++++++---------
vm.c | 1 -
vm_core.h | 4 ++--
4 files changed, 48 insertions(+), 12 deletions(-)
Updated by usa (Usaku NAKAMURA) over 8 years ago
- Backport changed from 2.2: UNKNOWN, 2.3: UNKNOWN, 2.4: UNKNOWN to 2.2: UNKNOWN, 2.3: UNKNOWN, 2.4: REQUIRED
Updated by usa (Usaku NAKAMURA) over 8 years ago
- Backport changed from 2.2: UNKNOWN, 2.3: UNKNOWN, 2.4: REQUIRED to 2.2: UNKNOWN, 2.3: DONE, 2.4: REQUIRED
Updated by nagachika (Tomoyuki Chikanaga) over 8 years ago
- Backport changed from 2.2: UNKNOWN, 2.3: DONE, 2.4: REQUIRED to 2.2: UNKNOWN, 2.3: DONE, 2.4: DONE
ruby_2_4 r59286 merged revision(s) 58284,58812,59028.
Updated by nvashchenko (Nikolay Vashchenko) over 8 years ago
I created a gem with a temporary workaround for versions 2.2.7, 2.3.4 and 2.4.1 to help folks until versions with backports are released:
https://rubygems.org/gems/stopgap_13632