Bug #20158
open
Ractor affects Coverage results
Description
I have a large rspec test suite. I found that if I call a Ractor, the Coverage results are strongly affected, i.e. almost all files appear to be uncovered. This happens even if I only ever call a Ractor before the library or rspec are required.
Unfortunately, I was not able to build a simple repro yet.
I assume it is a timing thing and only affects larger suites, or it only happens if there are multiple files, and maybe if the library lazily requires its sub-modules?
However, I guess this should produce the same results when added to the spec_helper.rb of other large suites:
# Ractor.new { nil } # uncomment this to affect coverage results require 'coverage' Coverage.start # require library, set up rspec etc. RSpec.configuration.after(:suite) do # this number is greatly reduced and unstable when calling Ractor above p Coverage.result.values.sum { |arr| arr.sum(&:to_i) } end I had this problem in this library. The problem affects simplecov users as well, as described here.
Updated by mame (Yusuke Endoh) almost 2 years ago
@janosch-x Could you explain the complete reproduction procedure? I couldn't reproduce the issue by the following configuration.
test.rb
Ractor.new { nil } require "coverage" Coverage.start load "test2.rb" foo pp Coverage.result test2.rb
def foo 1 end $ ruby test.rb test.rb:1: warning: Ractor is experimental, and the behavior may change in future versions of Ruby! Also there are many implementation issues. {"test2.rb"=>[1, 1, nil]} Indeed, coverage does not support Ractor (#20167). However, I don't understand why just creating a Ractor affects coverage.
Updated by jeremyevans0 (Jeremy Evans) almost 2 years ago
- Has duplicate Bug #20167: Code execution isn't recorded in Ractor added
Updated by janosch-x (Janosch Müller) almost 2 years ago
As mentioned in the ticket, i could not reproduce it with a smaller setup.
Maybe problems only begin at a certain size, or when there is some require hierarchy?
I've now forked my affected repository to demonstrate the problem: https://github.com/jaynetics/ractor_coverage_repro
Maybe you have some idea how to debug it based on this?
Updated by mame (Yusuke Endoh) almost 2 years ago
- Status changed from Open to Assigned
- Assignee set to ko1 (Koichi Sasada)
Thanks, I reproduce the problem successfully with rspec + Ractor + TracePoint (without coverage).
# test_spec.rb if ENV["RACTOR"] == "1" Ractor.new { nil } p :ractor_enabled else p :ractor_disabled end $tp = TracePoint.new(:line) {|t| pp t } $tp.enable describe "foo" do $tp.disable end Expected behavior: The last output should be the line of $tp.disable as follows.
$ RACTOR=0 rspec t_spec.rb :ractor_disabled ... #<TracePoint:line /home/mame/.rbenv/versions/ruby-dev/lib/ruby/gems/3.4.0+0/gems/rspec-core-3.12.2/lib/rspec/core/example_group.rb:398 in `subclass'> #<TracePoint:line /home/mame/work/ractor_coverage_repro/t_spec.rb:12> #<TracePoint:line <internal:trace_point>:297 in `disable'> No examples found. Finished in 0.00022 seconds (files took 7.97 seconds to load) 0 examples, 0 failures Actual result: Once Ractor mode is enabled, TracePoint stops firing in the middle of the process.
$ RACTOR=1 rspec t_spec.rb :ractor_enabled ... #<TracePoint:line /home/mame/.rbenv/versions/ruby-dev/lib/ruby/3.4.0+0/rubygems/basic_specification.rb:208 in `internal_init'> #<TracePoint:line /home/mame/.rbenv/versions/ruby-dev/lib/ruby/3.4.0+0/rubygems/basic_specification.rb:209 in `internal_init'> #<TracePoint:line /home/mame/.rbenv/versions/ruby-dev/lib/ruby/3.4.0+0/rubygems/stub_specification.rb:73 in `initialize'> #<TracePoint:line /home/mame/.rbenv/versions/ruby-dev/lib/ruby/3.4.0+0/rubygems/stub_specification.rb:74 in `initialize'> No examples found. Finished in 0.00027 seconds (files took 0.101 seconds to load) 0 examples, 0 failures Assigning to @ko1 (Koichi Sasada)
Updated by luke-gru (Luke Gruber) almost 2 years ago
It sounds like this bug is related to https://bugs.ruby-lang.org/issues/19112
Updated by jhawthorn (John Hawthorn) 6 months ago
- Assignee changed from ko1 (Koichi Sasada) to ractor