Other segfault on TestMarshal#test_gc

$ make test-all TESTS='-n !/callcc/' # and I edited test/ruby/test_continuation.rb to no-op CC = clang LD = ld LDSHARED = clang -shared CFLAGS = -O3 -fno-fast-math -ggdb3 -Wall -Wextra -Wno-unused-parameter -Wno-parentheses -Wno-long-long -Wno-missing-field-initializers -Wno-tautological-compare -Wno-parentheses-equality -Wno-constan t-logical-operand -Wno-self-assign -Wunused-variable -Werror=implicit-int -Werror=pointer-arith -Werror=write-strings -Werror=declaration-after-statement -Werror=shorten-64-to-32 -Werror=implicit-function-de claration -Werror=division-by-zero -Werror=deprecated-declarations -Werror=extra-tokens -std=gnu99 XCFLAGS = -D_FORTIFY_SOURCE=2 -fstack-protector -fno-strict-overflow -fvisibility=hidden -DRUBY_EXPORT -fPIE CPPFLAGS = -I. -I.ext/include/x86_64-linux -I../include -I.. -I../enc/unicode/9.0.0 DLDFLAGS = -Wl,--compress-debug-sections=zlib -fstack-protector -pie SOLIBS = -lgmp clang version 3.8.0-2ubuntu4 (tags/RELEASE_380/final) Target: x86_64-pc-linux-gnu Thread model: posix InstalledDir: /usr/bin Found candidate GCC installation: /usr/bin/../lib/gcc/i686-linux-gnu/6.0.0 Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/5.4.0 Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/6.0.0 Found candidate GCC installation: /usr/lib/gcc/i686-linux-gnu/6.0.0 Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/5.4.0 Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/6.0.0 Selected GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/5.4.0 Candidate multilib: .;@m64 Candidate multilib: 32;@m32 Candidate multilib: x32;@mx32 Selected multilib: .;@m64 Run options: "--ruby=./miniruby -I../lib -I. -I.ext/common ../tool/runruby.rb --extout=.ext -- --disable-gems" --excludes-dir=../test/excludes --name=!/memory_leak/ -n !/callcc/ [ 9354/16782] TestMarshal#test_gc/home/yura/Project/ruby-falcon/test/ruby/test_marshal.rb:181: [BUG] Segmentation fault at 0x0000000000000c ruby 2.4.0dev (2016-11-03 trunk 56550) [x86_64-linux] -- Control frame information ----------------------------------------------- c:0029 p:---- s:0167 e:000166 CFUNC :start c:0028 p:0019 s:0163 e:000162 METHOD /home/yura/Project/ruby-falcon/test/ruby/test_marshal.rb:181 [FINISH] c:0027 p:---- s:0158 e:000157 CFUNC :dump c:0026 p:0020 s:0153 e:000152 BLOCK /home/yura/Project/ruby-falcon/test/ruby/test_marshal.rb:188 c:0025 p:0062 s:0150 e:000149 METHOD /home/yura/Project/ruby-falcon/test/lib/test/unit/assertions.rb:177 c:0024 p:0009 s:0139 e:000138 METHOD /home/yura/Project/ruby-falcon/test/ruby/test_marshal.rb:187 c:0023 p:0036 s:0135 e:000134 METHOD /home/yura/Project/ruby-falcon/test/lib/test/unit.rb:1029 c:0022 p:0078 s:0129 e:000128 METHOD /home/yura/Project/ruby-falcon/test/lib/minitest/unit.rb:1269 c:0021 p:0022 s:0120 e:000119 METHOD /home/yura/Project/ruby-falcon/test/lib/test/unit/testcase.rb:18 c:0020 p:0073 s:0115 e:000114 BLOCK /home/yura/Project/ruby-falcon/test/lib/minitest/unit.rb:941 [FINISH] c:0019 p:---- s:0108 e:000107 CFUNC :map c:0018 p:0129 s:0104 E:001d80 METHOD /home/yura/Project/ruby-falcon/test/lib/minitest/unit.rb:934 c:0017 p:0054 s:0092 E:0021b8 METHOD /home/yura/Project/ruby-falcon/test/lib/test/unit.rb:914 c:0016 p:0016 s:0085 E:001750 BLOCK /home/yura/Project/ruby-falcon/test/lib/test/unit.rb:494 [FINISH] c:0015 p:---- s:0079 e:000078 CFUNC :each c:0014 p:0073 s:0075 E:000218 METHOD /home/yura/Project/ruby-falcon/test/lib/test/unit.rb:492 c:0013 p:0014 s:0068 E:0013f8 METHOD /home/yura/Project/ruby-falcon/test/lib/test/unit.rb:530 c:0012 p:0144 s:0061 E:001d88 METHOD /home/yura/Project/ruby-falcon/test/lib/minitest/unit.rb:885 c:0011 p:0010 s:0050 E:000eb0 METHOD /home/yura/Project/ruby-falcon/test/lib/minitest/unit.rb:1096 c:0010 p:0010 s:0046 E:000e48 BLOCK /home/yura/Project/ruby-falcon/test/lib/minitest/unit.rb:1083 [FINISH] c:0009 p:---- s:0042 e:000041 CFUNC :each c:0008 p:0057 s:0038 E:000ce8 METHOD /home/yura/Project/ruby-falcon/test/lib/minitest/unit.rb:1082 c:0007 p:0020 s:0033 E:000608 METHOD /home/yura/Project/ruby-falcon/test/lib/minitest/unit.rb:1070 c:0006 p:0012 s:0028 E:000000 METHOD /home/yura/Project/ruby-falcon/test/lib/test/unit.rb:682 c:0005 p:0022 s:0022 E:002598 METHOD /home/yura/Project/ruby-falcon/test/lib/test/unit.rb:33 c:0004 p:0045 s:0017 E:001f50 METHOD /home/yura/Project/ruby-falcon/test/lib/test/unit.rb:991 c:0003 p:0015 s:0013 E:001f08 METHOD /home/yura/Project/ruby-falcon/test/lib/test/unit.rb:995 c:0002 p:0304 s:0008 E:000ec8 EVAL ../test/runner.rb:40 [FINISH] c:0001 p:0000 s:0003 E:0026e0 (none) [FINISH] -- Ruby level backtrace information ---------------------------------------- ../test/runner.rb:40:in `<main>' /home/yura/Project/ruby-falcon/test/lib/test/unit.rb:995:in `run' /home/yura/Project/ruby-falcon/test/lib/test/unit.rb:991:in `run' /home/yura/Project/ruby-falcon/test/lib/test/unit.rb:33:in `run' /home/yura/Project/ruby-falcon/test/lib/test/unit.rb:682:in `run' /home/yura/Project/ruby-falcon/test/lib/minitest/unit.rb:1070:in `run' /home/yura/Project/ruby-falcon/test/lib/minitest/unit.rb:1082:in `_run' /home/yura/Project/ruby-falcon/test/lib/minitest/unit.rb:1082:in `each' /home/yura/Project/ruby-falcon/test/lib/minitest/unit.rb:1083:in `block in _run' /home/yura/Project/ruby-falcon/test/lib/minitest/unit.rb:1096:in `run_tests' /home/yura/Project/ruby-falcon/test/lib/minitest/unit.rb:885:in `_run_anything' /home/yura/Project/ruby-falcon/test/lib/test/unit.rb:530:in `_run_suites' /home/yura/Project/ruby-falcon/test/lib/test/unit.rb:492:in `_run_suites' /home/yura/Project/ruby-falcon/test/lib/test/unit.rb:492:in `each' /home/yura/Project/ruby-falcon/test/lib/test/unit.rb:494:in `block in _run_suites' /home/yura/Project/ruby-falcon/test/lib/test/unit.rb:914:in `_run_suite' /home/yura/Project/ruby-falcon/test/lib/minitest/unit.rb:934:in `_run_suite' /home/yura/Project/ruby-falcon/test/lib/minitest/unit.rb:934:in `map' /home/yura/Project/ruby-falcon/test/lib/minitest/unit.rb:941:in `block in _run_suite' /home/yura/Project/ruby-falcon/test/lib/test/unit/testcase.rb:18:in `run' /home/yura/Project/ruby-falcon/test/lib/minitest/unit.rb:1269:in `run' /home/yura/Project/ruby-falcon/test/lib/test/unit.rb:1029:in `run_test' /home/yura/Project/ruby-falcon/test/ruby/test_marshal.rb:187:in `test_gc' /home/yura/Project/ruby-falcon/test/lib/test/unit/assertions.rb:177:in `assert_nothing_raised' /home/yura/Project/ruby-falcon/test/ruby/test_marshal.rb:188:in `block in test_gc' /home/yura/Project/ruby-falcon/test/ruby/test_marshal.rb:188:in `dump' /home/yura/Project/ruby-falcon/test/ruby/test_marshal.rb:181:in `_dump' /home/yura/Project/ruby-falcon/test/ruby/test_marshal.rb:181:in `start' -- Machine register context ------------------------------------------------ RIP: 0x00007feedcfcf4dc RBP: 0x0000558c05d8f9c0 RSP: 0x00007ffc2be3ac20 RAX: 0x0000558c05d8ef20 RBX: 0x0000000000000001 RCX: 0x0000000000000004 RDX: 0x0000000000000004 RDI: 0x0000000000000014 RSI: 0x0000000000000001 R8: 0x0000558c08857780 R9: 0x0000000000000000 R10: 0x0000000000000003 R11: 0x0000000000000000 R12: 0x0000000000000014 R13: 0x0000000000000000 R14: 0x0000558c05d8f9c0 R15: 0x0000558c05fbea50 EFL: 0x0000000000010297 -- C level backtrace information ------------------------------------------- /home/yura/Project/ruby-falcon/.bld-clang~/ruby(rb_vm_bugreport+0x1cd) [0x558c04cd952d] ../vm_dump.c:679 /home/yura/Project/ruby-falcon/.bld-clang~/ruby(rb_bug_context+0x1e6) [0x558c04ccc466] ../error.c:426 /home/yura/Project/ruby-falcon/.bld-clang~/ruby(sigsegv+0x4f) [0x558c04bea17f] ../signal.c:897 /lib/x86_64-linux-gnu/libpthread.so.0 [0x7feeddcea3e0] /lib/x86_64-linux-gnu/libc.so.6(malloc_usable_size+0x1c) [0x7feedcfcf4dc] malloc.c:4584 /home/yura/Project/ruby-falcon/.bld-clang~/ruby(obj_free+0x224) [0x558c04b345f4] ../gc.c:7678 /home/yura/Project/ruby-falcon/.bld-clang~/ruby(gc_sweep_step+0x205) [0x558c04b33e15] ../gc.c:3451 /home/yura/Project/ruby-falcon/.bld-clang~/ruby(gc_sweep+0xab) [0x558c04b3575b] ../gc.c:3671 /home/yura/Project/ruby-falcon/.bld-clang~/ruby(gc_start+0x949) [0x558c04b338b9] ../gc.c:5602 /home/yura/Project/ruby-falcon/.bld-clang~/ruby(garbage_collect+0xb4) [0x558c04b2be84] ../gc.c:6306 /home/yura/Project/ruby-falcon/.bld-clang~/ruby(gc_start_internal+0x1c8) [0x558c04b30a58] ../gc.c:6623 /home/yura/Project/ruby-falcon/.bld-clang~/ruby(vm_call_cfunc+0xf1) [0x558c04c5a0c1] ../vm_insnhelper.c:1752 /home/yura/Project/ruby-falcon/.bld-clang~/ruby(vm_exec_core+0x2f1e) [0x558c04c4470e] ../insns.def:1066 /home/yura/Project/ruby-falcon/.bld-clang~/ruby(vm_exec+0x9d) [0x558c04c5486d] ../vm.c:1711 /home/yura/Project/ruby-falcon/.bld-clang~/ruby(rb_call0+0x17a) [0x558c04c6254a] ../vm_eval.c:61 /home/yura/Project/ruby-falcon/.bld-clang~/ruby(w_object+0x3c2) [0x558c04b5de92] ../marshal.c:727 /home/yura/Project/ruby-falcon/.bld-clang~/ruby(w_object+0x19ab) [0x558c04b5f47b] ../marshal.c:853 /home/yura/Project/ruby-falcon/.bld-clang~/ruby(rb_marshal_dump_limited+0x16d) [0x558c04b5d71d] ../marshal.c:1035 /home/yura/Project/ruby-falcon/.bld-clang~/ruby(marshal_dump+0xff) [0x558c04b5d53f] ../marshal.c:1004 /home/yura/Project/ruby-falcon/.bld-clang~/ruby(vm_call_cfunc+0xf1) [0x558c04c5a0c1] ../vm_insnhelper.c:1752 /home/yura/Project/ruby-falcon/.bld-clang~/ruby(vm_exec_core+0x2f1e) [0x558c04c4470e] ../insns.def:1066 /home/yura/Project/ruby-falcon/.bld-clang~/ruby(vm_exec+0x9d) [0x558c04c5486d] ../vm.c:1711 /home/yura/Project/ruby-falcon/.bld-clang~/ruby(invoke_block_from_c_splattable+0x158) [0x558c04c626d8] ../vm.c:1032 /home/yura/Project/ruby-falcon/.bld-clang~/ruby(rb_yield+0x8a) [0x558c04c4f62a] ../vm.c:1069 /home/yura/Project/ruby-falcon/.bld-clang~/ruby(rb_ary_collect+0x229) [0x558c04c725c9] ../array.c:2732 /home/yura/Project/ruby-falcon/.bld-clang~/ruby(vm_call_cfunc+0xf1) [0x558c04c5a0c1] ../vm_insnhelper.c:1752 /home/yura/Project/ruby-falcon/.bld-clang~/ruby(vm_exec_core+0x27fe) [0x558c04c43fee] ../insns.def:967 /home/yura/Project/ruby-falcon/.bld-clang~/ruby(vm_exec+0x9d) [0x558c04c5486d] ../vm.c:1711 /home/yura/Project/ruby-falcon/.bld-clang~/ruby(invoke_block_from_c_splattable+0x158) [0x558c04c626d8] ../vm.c:1032 /home/yura/Project/ruby-falcon/.bld-clang~/ruby(rb_yield+0x8a) [0x558c04c4f62a] ../vm.c:1069 /home/yura/Project/ruby-falcon/.bld-clang~/ruby(rb_ary_each+0x29) [0x558c04c6c939] ../array.c:1823 /home/yura/Project/ruby-falcon/.bld-clang~/ruby(vm_call_cfunc+0xf1) [0x558c04c5a0c1] ../vm_insnhelper.c:1752 /home/yura/Project/ruby-falcon/.bld-clang~/ruby(vm_exec_core+0x27fe) [0x558c04c43fee] ../insns.def:967 /home/yura/Project/ruby-falcon/.bld-clang~/ruby(vm_exec+0x9d) [0x558c04c5486d] ../vm.c:1711 /home/yura/Project/ruby-falcon/.bld-clang~/ruby(invoke_block_from_c_splattable+0x158) [0x558c04c626d8] ../vm.c:1032 /home/yura/Project/ruby-falcon/.bld-clang~/ruby(rb_yield+0x8a) [0x558c04c4f62a] ../vm.c:1069 /home/yura/Project/ruby-falcon/.bld-clang~/ruby(rb_ary_each+0x29) [0x558c04c6c939] ../array.c:1823 /home/yura/Project/ruby-falcon/.bld-clang~/ruby(vm_call_cfunc+0xf1) [0x558c04c5a0c1] ../vm_insnhelper.c:1752 /home/yura/Project/ruby-falcon/.bld-clang~/ruby(vm_exec_core+0x27fe) [0x558c04c43fee] ../insns.def:967 /home/yura/Project/ruby-falcon/.bld-clang~/ruby(vm_exec+0x9d) [0x558c04c5486d] ../vm.c:1711 /home/yura/Project/ruby-falcon/.bld-clang~/ruby(ruby_run_node+0x138) [0x558c04b1ba28] ../eval.c:244 /home/yura/Project/ruby-falcon/.bld-clang~/ruby(main+0x4e) [0x558c04b1a16e] ../main.c:36 -- Other runtime information ----------------------------------------------- * Loaded script: ../test/runner.rb: TestMarshal#test_gc ..... 

Seems related to segfaults on FreeBSD 11.
I tried with MacPorts clang 3.8.1, but it didn't reproduce.
Maybe clang 3.8.0's issue?

FreeBSD failure log is here:

http://rubyci.org/logs/rubyci.s3.amazonaws.com/freebsd11zfs/ruby-trunk/log/20161104T063002Z.fail.html.gz

Reproduced on my FreeBSD box:

./ruby -r continuation -e 'cont = nil; ary = [1,2,3]; ary.combination(2) { callcc {|k| cont = k} unless cont }; cont.call' ruby 2.4.0dev (2016-11-04 trunk 56558) [x86_64-freebsd11.0] -- Control frame information ----------------------------------------------- c:0005 p:---- s:0020 e:000019 CFUNC :callcc c:0004 p:0016 s:0016 E:002420 BLOCK -e:1 [FINISH] c:0003 p:---- s:0013 e:000012 CFUNC :combination c:0002 p:0017 s:0008 E:001850 EVAL -e:1 [FINISH] c:0001 p:0000 s:0003 E:002700 (none) [FINISH] ... 

In cont_capture(), cont is set to 0x1 after the context is restored.

 if (ruby_setjmp(cont->jmpbuf)) { /* cont is set to 0x1 here */ 

The following patch fixes the problem:

diff --git a/cont.c b/cont.c index 8119b5c..79b685f 100644 --- a/cont.c +++ b/cont.c @@ -471,6 +471,9 @@ cont_new(VALUE klass) static VALUE cont_capture(volatile int *stat) +#ifdef __clang__ +__attribute__ ((optnone)) +#endif  { rb_context_t *cont; rb_thread_t *th = GET_THREAD(); 

Shugo Maeda wrote:

static VALUE
cont_capture(volatile int *stat)
+#ifdef clang
+attribute ((optnone))
+#endif

I'm not sure whether the version of clang should be checked as follows:

diff --git a/cont.c b/cont.c index 8119b5c..85d3766 100644 --- a/cont.c +++ b/cont.c @@ -471,6 +471,10 @@ cont_new(VALUE klass) static VALUE cont_capture(volatile int *stat) +#if defined(__clang__) && \ + __clang_major__ == 3 && __clang_minor__ == 8 && __clang_patch__ == 0 +__attribute__ ((optnone)) +#endif  { rb_context_t *cont; rb_thread_t *th = GET_THREAD(); 

NetBSD pkgsrc and FreeBSD ports were fixed by adding volatile.
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=206111

Yui NARUSE wrote:

NetBSD pkgsrc and FreeBSD ports were fixed by adding volatile.
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=206111

It seems to be a different issue.

I tried adding volatile to cont in cont_new(), but it didn't work.

Shugo Maeda wrote:

Yui NARUSE wrote:

NetBSD pkgsrc and FreeBSD ports were fixed by adding volatile.
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=206111

It seems to be a different issue.

I tried adding volatile to cont in cont_new(), but it didn't work.

Did you add it as volatile rb_context_t *cont, or as rb_context_t *volatile cont? Only the latter is correct.

Please refer to #13014 for the full patch set, including fix for cont_new().

Dimitry Andric wrote:

NetBSD pkgsrc and FreeBSD ports were fixed by adding volatile.
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=206111

It seems to be a different issue.

I tried adding volatile to cont in cont_new(), but it didn't work.

Did you add it as volatile rb_context_t *cont, or as rb_context_t *volatile cont? Only the latter is correct.

Please refer to #13014 for the full patch set, including fix for cont_new().

Adding the latter version to cont_capture() didn't work, but your full patch set did work.
Thank you!