Bug #13164
open
A second `SystemStackError` exception results in `Segmentation fault (core dumped)`
Description
This issue is was exposed by leveraging the fact that Object#hash is implemented recursively for core Ruby datatypes (i.e., Hash and Array). See the discussion here: https://github.com/boazsegev/combine_pdf/pull/91#issuecomment-275552131.
TO reproduce the issue, explode the stack twice.
Expected results:
SystemStackError will be raised both times.
Actual results:
SystemStackError is raised once. The second time will cause a core dump.
Code to cause core dump:
def compute_nest_depth h = {nest: {}} nest = h[:nest] i = 0 while true i += 1 puts "nested #{i}" if ((i & 511) == 0) next_nest = { nest: {} } nest[:nest] = next_nest nest = next_nest[:nest] h.hash end rescue SystemStackError puts "Stack exploded at nesting #{i}" end counter = 0; while(true) begin counter +=1 puts "starting test #{counter}" compute_nest_depth rescue SystemStackError => e nil ensure puts "test #{counter} complete" end end results:
starting test 1 nested 512 nested 1024 nested 1536 nested 2048 nested 2560 Stack exploded at nesting 2783 test 1 complete starting test 2 nested 512 nested 1024 nested 1536 nested 2048 nested 2560 Segmentation fault (core dumped) 
Updated by nobu (Nobuyoshi Nakada) almost 9 years ago
By doubling rb_sigaltstack_size(), it doesn't segfault and the second or more stack overflows never happen now.
I suspect that the stack guard page may need to be reset, but not sure.
diff --git a/signal.c b/signal.c index 888c8eaa72..8947b0ea95 100644 --- a/signal.c +++ b/signal.c @@ -563,7 +563,7 @@ rb_sigaltstack_size(void) } #endif - return size; + return size * 2; } /* alternate stack for SIGSEGV */ 
Updated by myst (Boaz Segev) almost 9 years ago
This is a good observation and I'm happy you found this...
However, I'm not sure that using return size * 2 as a patch will solve the issue. It might end up masking the real issue, making it harder to find (although I might be wrong).
At the moment, there is a segmentation fault. Is it possible that the size returned is somehow effecting a memory address / pointer in a way that it shouldn't...?
Updated by nobu (Nobuyoshi Nakada) almost 9 years ago
When configured with --with-setjmp-type=sigsetjmp, it seemed working.
But segfaulted at the fourth system stack overflow.
Updated by myst (Boaz Segev) almost 9 years ago
What about flattening recursion in core types (Hash, Array and Set)?
I know this won't resolve the issue, but it will prevent eql? and hash from exploding the stack, so the issue is less likely to occur when there isn't an error in the code being executed.
Updated by nobu (Nobuyoshi Nakada) over 8 years ago
- Has duplicate Bug #13412: Infinite recursion with define_method may cause silent SEGV or cfp consistency error added
Updated by nobu (Nobuyoshi Nakada) over 8 years ago
- Has duplicate deleted (Bug #13412: Infinite recursion with define_method may cause silent SEGV or cfp consistency error)
Updated by nobu (Nobuyoshi Nakada) over 8 years ago
- Related to Bug #13412: Infinite recursion with define_method may cause silent SEGV or cfp consistency error added
Updated by nobu (Nobuyoshi Nakada) over 8 years ago
- Status changed from Open to Closed
Applied in changeset trunk|r58353.
signal.c: unblock signal
- signal.c (raise_stack_overflow): unblock the received signal, to
receive the same signal again. [ruby-core:79285] [Bug #13164]
Updated by nobu (Nobuyoshi Nakada) over 8 years ago
- Status changed from Closed to Open
On Linux, fixed by unblocking the received signal.
But it has no effect on mac OS and seems to need --with-setjmp-type=setjmp.
Updated by nobu (Nobuyoshi Nakada) over 8 years ago
- Has duplicate Bug #13596: Segfault when catching SystemStackError in eval added
Updated by shyouhei (Shyouhei Urabe) about 8 years ago
- Related to Bug #13948: Segfault instead of recursion depth error added