I've submitted a pull request for an updated version of @normalperson's patch: https://github.com/ruby/ruby/pull/5002, expanding the optimization to include nil/true/false/0-9. This is an across-the-board performance improvement, mostly...jeremyevans0 (Jeremy Evans)
It seems like `gc_verify_compaction_references` is not protected in case alignment is wrong. This commit pushes the alignment check down to `gc_start_internal` so that anyone trying to compact will check page alignment I think this met...tenderlovemaking (Aaron Patterson)
We can look into this. The downside is that YJIT does a lot of code patching, so this would presumably involve two system calls every time YJIT needs to generate code. Hopefully the performance impact will be relatively minimal.maximecb (Maxime Chevalier-Boisvert)
YJIT uses `mmap` with `PROT_READ | PROT_WRITE | PROT_EXEC` protections, which reduces security because it makes code injection attacks much easier, as the attacker only needs to be able to write their shellcode to the mapped memory regio...jeremyevans0 (Jeremy Evans)
2d98593bf54a37397c6e4886ccc7e3654c2eaf85 made it so that attr_accessor methods fire C method tracing events. Previously, we weren't checking for whether we are tracing before compiling, leading to missed events. Since global invalidatio...alanwu (Alan Wu)
I am not sure why this flag was turned off (it wasn't explained in my commit message in 0365dc852767ae589376a7aad1fb129738e408b0 or in my PR in #4411). Whatever the reason, without `default_ignores` turned on, most default CI configurat...searls (Justin Searls)
TestRubyOptions#test_enable was broken on OpenBSD after the yjit merge. --yjit (and --enable-all, which enables --yjit) fails on OpenBSD because yjit uses an insecure mmap call (both writable and executable), in alloc_exec_mem, which Ope...jeremyevans (Jeremy Evans)
vo.x (Vit Ondruch) wrote in #note-2: > ~~~ > ... The `flags` 73831 = 0x12067, that means `orig` is an `Array` object, which has two elements(0x00000001000728e8, 0x0000000100072870).nobu (Nobuyoshi Nakada)
nobu (Nobuyoshi Nakada) wrote in #note-3: > Found the bug. > ... Here I am. Apparently it doesn't. I dug a bit deeper and I found that there are two issues that concur to this behavior: * `ioctl_narg_len` isn't properly extracti...vihai (Daniele Orlandi)
Proposal to RuboCop to introduce warning: https://github.com/rubocop/rubocop/issues/10204 I support deprecating `$!` and `$@`. One way: for Ruby 3.x we can create deprecation warning, and remove it in Ruby 4.x.ioquatix (Samuel Williams)
ko1 (Koichi Sasada) wrote in #note-20: > Sorry, my proposal is introducing dynamic scope, not syntactical scope like: Ah, I misunderstood. Then I think deprecating `$!` is the right way, since that seems the only way to be able to...Eregon (Benoit Daloze)
First, I am not going to change the syntax for the `ensure` clause. The `ensure` clause is to unwind-protect so it is not recommended to branch depending on exceptions. I admit the current `$!` behavior is error-prone. But considering...matz (Yukihiro Matsumoto)
Sorry, my proposal is introducing dynamic scope, not syntactical scope like: ```ruby def foo p $! #=> A begin # no raise ensure p $! #=> nil end end begin raise A ensure foo end ``` We can implement `raise` like me...ko1 (Koichi Sasada)
I prepared versions for Ruby 3.1.0-preview1. But I keep the version numbers of the following libraries. * RubyGems/Bundler: I coordinate the version numbering with @deivid * Reline, Irb: @aycabta * BIgDecimal: @mrkn * Fiddle, CSV, Strin...hsbt (Hiroshi SHIBATA)
When we introduce a new specifier for strftime, we are afraid that we accidentally introduce it as a different meaning from the de-facto standard. Maybe you insist coreutil's date is an implementation near the de-facto, but I think the...naruse (Yui NARUSE)
Re naming, I discussed with @ioquatix and I think this is consistent and makes sense: ```ruby class MyClass extend Immutable end MyClass.new.immutable? # true MY_CONSTANT = Immutable [1, 2, [3]] MY_CONSTANT.immutable? # tr...Eregon (Benoit Daloze)
Use cases: * Actually have a way to express immutable/deeply frozen in Ruby. Right now there is no way, and `freeze` is only shallow. * Immutability has many benefits, easier to reason about, no data races possible, clarifies a lot the b...Eregon (Benoit Daloze)
The above sounds good. Note that `RB_FL_FREEZE | RB_FL_SHAREABLE` is not enough, it should be a new flag, I detailed in https://bugs.ruby-lang.org/issues/18035#note-24. Re (2) I think `.deep_freeze` would be a good option too. `Immut...Eregon (Benoit Daloze)
After discussing it, we found some good ideas regarding immutability in Ruby. Firstly, we need some definition for immutability. There appear to be two options. (1) We can assume that `RB_FL_FREEZE | RB_FL_SHAREABLE` implies immutable ...ioquatix (Samuel Williams)
We have discussed this in today's developer meeting. - As a tentative workaround, I'll fix the digest gem not to try to install `digest.h` to an unwritable directory so `gem install --user digest` succeeds. - It is (technically) poss...knu (Akinori MUSHA)
I think for the semantic model it could be easier with the proposed semantics in this issue: only always-shareable objects are shareable without `Ractor.make_shareable`/`freeze` for leaf objects with no ivars/`deep_freeze/Immutable`. Ri...Eregon (Benoit Daloze)
I feel from a predictability POV, it would definitely be advantageous to set `RB_FL_SHAREABLE` in every case it's known ahead of time to avoid unexpected overheads. Even if `Ractor.shareable?` is not public interface, it's still used int...ioquatix (Samuel Williams)
Yeah that's an alternative design. Currently `Ractor.shareable?` semantics are "is it already shareable as in conceptually or not?". (And the flag is just a cache for the "yes" case) And it's not "is it already shareable as marked w...Eregon (Benoit Daloze)
On my computer, even with a relatively small object graph,`Ractor.shareable?` can be quite slow (around 1-2ms). The following example creates an object graph with ~40k objects as an example, and on my computer takes around 20ms to execut...ioquatix (Samuel Williams)
Refinement#import_methods imports methods from modules. Unlike Module#include, it copies methods and adds them into the refinement, so the refinement is activated in the imported methods. [Bug #17429] [ruby-core:101639]shugo (Shugo Maeda)
`RubyVM.keep_script_lines` enables to keep script lines for each ISeq and AST. This feature is for debugger/REPL support. ```ruby RubyVM.keep_script_lines = true RubyVM::keep_script_lines = true eval("def foo = nil\ndef bar = nil") pp ...ko1 (Koichi Sasada)
I agree with merging it, but it's too close to the 3.1 release. So how about merging it, turned off by default, then turning it on after the release? Matz. matz (Yukihiro Matsumoto)
While f6048e592c520e20ba498ba4d553f07f02213a61 tried to approach the instability mentioned there, unfortunately the Travis arm32 environment seems unstable by itself. I already saw 2 different failures in the last 4 builds: https://app.t...k0kubun (Takashi Kokubun)
@alanwu has just merged https://github.com/ruby/ruby/pull/4992 at commit:6a9e2b3cc381fd1b6c5ec5eddb077ea5468ef75e. Congratulations!k0kubun (Takashi Kokubun)
Thank you Matz & Kokubun I will follow the instructions to register as a committer :) The upstreaming PR on ruby/ruby is now open. Currently working on a last-minute issue, trying to resolve a bug we found this morning, but otherwi...maximecb (Maxime Chevalier-Boisvert)
Thank you for your support, Matz! @maximecb Could you follow https://bugs.ruby-lang.org/projects/ruby/wiki/CommitterHowto#What-to-do-for-registering-you-as-a-committer to give commit access to you?k0kubun (Takashi Kokubun)
I accept @maximecb as a committer. Welcome! For the concrete process for merging YJIT into master, consult with @k0kubun. Matz. matz (Yukihiro Matsumoto)
znz (Kazuhiro NISHIYAMA) wrote in #note-5: > `'\bMutex\b'` counts `Thread::Mutex` too. True, although given the proportion it doesn't really change things much. `gem-codesearch` is always an approximation anyway. Trying to be a l...Eregon (Benoit Daloze)
On non RUBY_DEBUG builds, assert() compiles to nothing and the compiler warns about uninitialized variables in those code paths. Replace those asserts with rb_bug() to fix the warnings and do the assert in all builds. Since yjit_asm_test...alanwu (Alan Wu)
Previously, we were shuffling keyword arguments before checking for interrupts. In the case that we side exit in the interrupt check, we left the interpreter with an already-shuffled argument list for the call, resulting in a double shuf...alanwu (Alan Wu)
In an effort to simplify the logic YJIT generates for accessing instance variable, YJIT ensures that a given name-to-index mapping exists at compile time. In the case that the mapping doesn't exist, it was created by using rb_ivar_set() ...alanwu (Alan Wu)
When YJIT make calls to routines without reconstructing interpreter state through jit_prepare_routine_call(), it relies on the routine to never allocate, raise, and push/pop control frames. Comment about this on the routines that YJTI ca...alanwu (Alan Wu)
Previously, options such as "--yjit123" would enable YJIT. Additionally, the error message for argument parsing mentioned "--jit-..." instead of "--yjit-...".alanwu (Alan Wu)
This reverts commit 60f3f25cc421b75b3dbf02fef759c4857bf3ef46. We don't need to pass --disable-yjit when running MJIT tests anymore because we are off by default.alanwu (Alan Wu)
Since opt_getinlinecache and opt_setinlinecache point to the same cache struct, there is no need to track the index of the get instruction and then store it on the cache struct later when processing the set instruction. Setting it when p...alanwu (Alan Wu)
tool/test-bundled-gems.rb use sub processes for testing bundled gems and doesn't support RUN_OPTS. We weren't enabling YJIT for these tests. Use an include config with RUBY_YJIT_ENABLE to turn on YJIT for these tests. Note that we only ...alanwu (Alan Wu)
There might be code out there that expect `ruby -v` to print only one line. Since MJIT shows +JIT in `ruby -v` and RUBY_DESCRIPTION, let's show +YJIT. The crash report doesn't show anything about MJIT, so adjust the test. The "test_ruby...alanwu (Alan Wu)
The new backend isn't used at the moment and adds to our diff against upstream so remove it for now. We can reverse the removal later with git history.alanwu (Alan Wu)
We were exporting a couple of symbols in yjit.h because they could be used by code generated by MJIT. We don't want MJIT calling into YJIT code anyways so let's stop exporting them to libruby.so. Also adjust indentation and comments in ...alanwu (Alan Wu)
For upstreaming, we want functions we export either prefixed with "rb_" or made static. Historically we haven't been following this rule, so we were "leaking" a lot of symbols as `make leak-globals` would tell us. This change unifies ev...alanwu (Alan Wu)
We need to reconstruct interpreter state before calling into the routines to be able to raise exceptions. I'm getting a crash in debug build with: make test-all 'TESTS=test/ruby/variable.rb' RUN_OPTS='--yjit-call-threshold=1 --yjit-m...alanwu (Alan Wu)
This is so we know the specific reason we're exiting this instruction. Co-authored-by: Aaron Patterson tenderlove@ruby-lang.orgeileencodes (Eileen Uchitelle)
The old heap object check is not as efficient as the one in jit_guard_known_klass(). Also, the old code saves cfp->sp after popping the operands off the stack, which might cause the operands to be not marked by the GC in some circumstances.alanwu (Alan Wu)
When calling "to_s" on an instance of String, the method simply returns self. In this situation most of the work comes from setting up the method call. It turns out that both railsbench and liquid-render do this a lot. When generating c...alanwu (Alan Wu)
Before this change, when we encounter a constant cache that is specific to a lexical scope, we unconditionally exit. This change falls back to the interpreter's cache in this situation. This should help constant expressions in `class <<...alanwu (Alan Wu)
YJIT expects the VM to invalidate opt_getinlinecache when updating the constant cache, and the invalidation used to happen even when YJIT can't use the cached value. Once the first invalidation happens, the block for opt_getinlinecache ...alanwu (Alan Wu)
In vm_push_frame(), the stack overflow test is done against a decremented cfp. YJIT wasn't accounting for that in its stack overflow tests.alanwu (Alan Wu)
It's already up to date. We set it when we do iseq calls and it's set properly on entry. If it's not set properly exceptions wouldn't work.alanwu (Alan Wu)
Follow up for ecb5b383a0c17550b9b27663005049ddac871edb. Now that YJIT_STATS is defined in yjit.h, it shoudl be the only place that defines it.alanwu (Alan Wu)
We weren't counting completing an entire method in YJIT as exits so the avg_len_in_yjit for ./miniruby --yjit-call-threshold=1 --yjit-stats -e'def foo; end; foo' was infinite.alanwu (Alan Wu)
Send instructions currently generate the exact same side exit twice. Cache the exit the first time we generate it. Also add a comment explaining what side exits do. Closes GH-117.alanwu (Alan Wu)
The interpreter instruction count was enabled based on RUBY_DEBUG as opposed to YJIT_STATS. In builds with YJIT_STATS=1 but RUBY_DEBUG=0, the count was not available. Move YJIT_STATS in yjit.h where declarations are expoed to code outsi...alanwu (Alan Wu)
YJIT and MJIT can't be running in the same process otherwise they'll clobber each other. We should show an error and exit if they're both enabled.tenderlovemaking (Aaron Patterson)
We need to fire the write barrier during ivar set. This function extracts the write barrier function then calls it. Co-Authored-By: John Hawthorn <john@hawthorn.email>tenderlovemaking (Aaron Patterson)
We have a check to ensure we don't have to push args on the stack to call a cfunc with many args. However we never need to use the stack for variadic cfuncs, so we shouldn't care about the number of arguments.jhawthorn (John Hawthorn)
The code path for leave that returns to the interpreter (gen_leave() -> yjit_gen_leave_exit()) used to have the logic: ``` cfp->sp++; cfp->sp[-1] = return_val; cfp->sp--; return return_val; ``` The SP changes it made wa...alanwu (Alan Wu)
Previously checktype only supported heap objects, however it's not uncommon to receive an immediate, for example when string interpolating a Symbol or Integer.jhawthorn (John Hawthorn)
We should only clear the JIT function when the entry point is invalidated. Right now we only support compiling functions with a PC offset of zero (functions that take optional parameters can start at non-zero PC), so this patch just che...tenderlovemaking (Aaron Patterson)
For use cases where you want to collect the metrics for a specific piece of code (typically a web request) you can have the stats turned off by default and then turn them on at runtime before executing the code you care about.byroot (Jean Boussier)
This change fixes some cases where YJIT fails to fire tracing events. Most of the situations YJIT did not handle correctly involves enabling tracing while running inside generated code. A new operation to invalidate all generated code i...alanwu (Alan Wu)
RUBY_DEBUG have a very significant performance overhead. Enough that YJIT with RUBY_DEBUG is noticeably slower than the interpreter without RUBY_DEBUG. This makes it hard to collect yjit-stats in production environments. By allowing to...byroot (Jean Boussier)
The FIXME is there so we remember to investigate why insns clears the temporary array. Is this necessary? If it's not we can remove it from both. Co-authored-by: Aaron Patterson <tenderlove@ruby-lang.org>eileencodes (Eileen Uchitelle)
Previously all stack operands would become unknown once the stack grew beyond a certain size. This worked, but unnecessarily hid available information.jhawthorn (John Hawthorn)
If we hit this at PC > 0 (ie. with an optional argument) the provided types and context are likely incorrect and it is likely this block can't be used.jhawthorn (John Hawthorn)
Adds yjit support for setting global variables. Co-authored-by: Aaron Patterson <tenderlove@ruby-lang.org> Co-authored-by: John Hawthorn <john@hawthorn.email>eileencodes (Eileen Uchitelle)
Previously, under the scraper, this would side-exit because it was returning to a C method. Now that we use the jit_func entrypoint, this test no longer side-exits.jhawthorn (John Hawthorn)
Clear out any JIT code on iseqs when tracepoints get enabled. We can't handle tracepoints right now, so we'll just try to recompile later.tenderlovemaking (Aaron Patterson)
If `--disable-jit-support` is passed to configure, then `jit_func` is removed from the iseq body and we can't compile YJIT. This commit detects when the JIT function pointer is gone and disables YJIT in that case.tenderlovemaking (Aaron Patterson)
Now that we're using the jit function entry point, we don't need the scraper. Thank you for your service, scraper. ❤️tenderlovemaking (Aaron Patterson)
The JIT entry point passes the CFP as RSI and the EC as RDI. Lets match that so we don't have to shuffle registers around.tenderlovemaking (Aaron Patterson)
Methods with optional parameters don't always start executing at the first PC, but we compile all methods assuming that they do. This commit adds a guard to ensure that we're actually starting at the first PC for methods with optional p...tenderlovemaking (Aaron Patterson)
Always using `ret` to return to the interpreter means that we never have to check the VM_FRAME_FLAG_FINISH flag. In the case that we return `Qundef`, the interpreter will execute the cfp. We can take advantage of this by setting the PC...tenderlovemaking (Aaron Patterson)
We clear locals when we know their values might change (ex. when performing a method call). However previously values on the stack which were originally pushed from a local would still point back to that local. With this commit, when cl...jhawthorn (John Hawthorn)
This script is an lldb helper that just loops through all the comments stored and prints out the comment along with the address corresponding to the comment. For example, I'm crashing in JIT code at address 0x0000000110000168. Using the...tenderlovemaking (Aaron Patterson)
In jit_guard_known_klass whenever we encounter a new class should recompile the current instruction. However, previously once jit_guard_known_klass had guarded for a heap object it would not recompile for any immediate (special const) ...jhawthorn (John Hawthorn)
I'm looking through the places where YJIT needs notifications. It looks like these changes to gc.c and vm_callinfo.h have become unnecessary since 84ab77ba592. This commit just makes the diff against upstream smaller, but otherwise sho...tenderlovemaking (Aaron Patterson)
We recently added the ability to optimize a known cfunc with custom code generation for it. Previously we performed this lookup with a switch statement on the address of the c function being called. This commit swaps that out for a hash...jhawthorn (John Hawthorn)
This adds a method to blocks to get outgoing ids, then uses the outgoing ids to generate a graphviz graph. Two methods were added to the Block object. One method returns an id for the block, which is just the address of the underlying ...tenderlovemaking (Aaron Patterson)
As an optimization, multiple objects could share the same singleton class. The optimization introduced in 6698e433934d810b16ee3636b63974c0a75c07f0 wasn't handling this correctly so was generating guards that never pass for the inputs we ...alanwu (Alan Wu)
Singleton classes should only ever be attached to one object. This means that checking for the object should be the same as checking for the class. This should be slightly faster by avoiding one memory acccess as well as allowing us to s...jhawthorn (John Hawthorn)
jit_rb_obj_not() wants to access the type information of the receiver, but we were discarding type info of locals before jit_rb_obj_not() runs unncessarily. There are also cases we are unncessarily discarding local type info. For exampl...alanwu (Alan Wu)
This commit improves opt_not by making it correct when TrueClass#! and/or FalseClass#! is defined and genearting better code when the receiver is a heap object. guard_known_class() can now handle true, false, and nil, and we introduce a...alanwu (Alan Wu)
invokebuiltin_delegate can be run in place of invokebuiltin_delegate_leave because there is always a leave instruction afterwards (the interpreter takes advantage of this as well when rewriting iseqs for tracing).jhawthorn (John Hawthorn)
invokebuiltin_delegate is a special version of invokebuiltin used for sending a contiguous subset of the current method's locals. In some cases YJIT would already handle this for trivial cases it could be inlined, implementing this OP a...jhawthorn (John Hawthorn)
* Use builtin_inline_p to skip a frame of C methods * Fix bugs in primitive cfunc call code * Remove if (push_frame) {} * Remove if (push_frame) {} * Push Aaron's fix to avoid hardcoding insn lengths Co-authored-by: Takashi Kokubun ...Maxime Chevalier-Boisvert
* Implement gen_newarray * Implement newhash for n=0 * Add yjit tests for newhash/newarray * Fix integer size warning on clang * Save PC and SP in newhash and newarray Co-authored-by: Maxime Chevalier-Boisvert <maximechevalierb@gmai...jhawthorn (John Hawthorn)
Previously this could crash on Nokogiri when JITing the getivar instruction because we would attempt to treat Nokogiri::XML::Document's T_DATA as a T_OBJECT in calling rb_iv_index_tbl_lookup. This commit also checks for T_OBJECT at comp...jhawthorn (John Hawthorn)
* Implement getblockparamproxy * Parallel runner: wait for timeout thread to terminate after killing Or else the leak cheaker could sees the thread as running and cause test failures in test-tool. * Add a comment, use jne * Comment a...alanwu (Alan Wu)
* Implement send with blocks Not that much extra work compared to `opt_send_without_block`. Moved the stack over flow check because it could've exited after changes are made to cfp. * rename oswb counters * Might as well implement sen...alanwu (Alan Wu)
* Use rb_ivar_get() for general case of getivar Pretty straight forward. Buys about 1% coverage on railsbench. * Update yjit_codegen.c Co-authored-by: Maxime Chevalier-Boisvert <maximechevalierb@gmail.com>alanwu (Alan Wu)
* Implement calls to methods with simple optional params * Remove unnecessary MJIT_STATIC See comment for MJIT_STATIC. I added it not knowing whether it's required because the function next to it has it. Don't use it and wait for probl...alanwu (Alan Wu)
* Use INT32_MIN, INT32_MAX, etc. constants in yjit_asm.c * Print warning on stderr when code past rel32 jump range * Fix preprocessor snafu * Move rel32 warning into --yjit-stats * Try to allocate within rel32 offset on Linux machine...Maxime Chevalier-Boisvert
* Create GitHub workflow to run ASM tests * Specify path * Attempt #3 * Set bash flags to print commands, stop on first error * Remove clear command from test_asm.sh * Use clangMaxime Chevalier-Boisvert
This commit fixes a build error. If we build in release mode (IOW *without* RUBY_DEBUG), then this constant isn't defined. Release mode builds are required by yjit-benchtenderlovemaking (Aaron Patterson)
Make sure `opt_getinlinecache` is in a block all on its own, and invalidate it from the interpreter when `opt_setinlinecache`. It will recompile with a filled cache the second time around. This lets YJIT runs well when the IC for constan...alanwu (Alan Wu)
ASAN can catch these type of things for us, but the scraper can't handle ASAN :/. To be more resilient to refactoring, extend the lifetime of `generic_ctx` in branch_stub_hit() too.alanwu (Alan Wu)
Introduce a new macro `ADD_COMMENT(cb, comment)` that records a comment for the current write position in the code block. Co-authored-by: Maxime Chevalier-Boisvert <maximechevalierb@gmail.com> Co-authored-by: Aaron Patterson <aaron.patt...alanwu (Alan Wu)
This commit collects statistics about how many binding objects are allocated as well as the number of local variables set on bindings. Statistics are output along with other YJIT stats. Here is an example of the output: ``` ***YJIT: Pr...tenderlovemaking (Aaron Patterson)
The old code decides branch->shape based on the write position of the native code block, which is unsound in case the block already exists and no new code is written to the write position. Make this decision with the start address of th...alanwu (Alan Wu)
Replacing the key was only okay if the new key hashes to the same thing as the old key. That doesn't hold for YJIT's table when the keys move.alanwu (Alan Wu)
We didn't need the private check before because we were lifting from the interpreter's cache, and the interpreter only caches when visibility checks go through.alanwu (Alan Wu)
* YJIT: unify exits. Patch iseqs only when necessary This fixes the gotcha that returning YJIT_CANT_COPMILE for an instruction at entry position leading to infinite loop. Also, iseq patching is only done only when necessary, which shou...alanwu (Alan Wu)
This is playing it safe. Ruby headers rely on some macros generated by autoconf that affect what system headers behave. We should take care to bring in those configs before including systme headers to stay consistent with the rest of Ruby.alanwu (Alan Wu)
Lazily compile out a chain of checks for different known classes and whether `self` embeds its ivars or not. * Remove trailing whitespaces * Get proper addresss in Capstone disassembly * Lowercase address in Capstone disassembly Caps...alanwu (Alan Wu)
When a BOP is redefined, the BOP redefinition callback will invalidate any blocks that depend on BOPS. This allows us to eliminate runtime checks for BOP redefinition.tenderlovemaking (Aaron Patterson)
Also filter with not equal to zero instead of greater than zero, in case there is memory corruption and counters are negative. Fixes Shopify/ruby#70alanwu (Alan Wu)
For deferred compilation, we sometimes want to care about the order of the block versions. Use an array instead of a linked list to do that.alanwu (Alan Wu)
Filter out counters that are zero and make it easier to have multiple printout groups. On `railsbench`: ``` opt_send_without_block exit reasons: se_finish_frame 184809 (100.0%) ```alanwu (Alan Wu)
This allows counting things in generated code such as side exits or different reasons for failing to compile. Runtime counters are exposed to Ruby as `UJIT.runtime_stats` for ease of reporting and analysis.alanwu (Alan Wu)
* ujit: implement opt_getinlinecache Aggressively bet that writes to constants don't happen and invalidate all opt_getinlinecache blocks on any and all constant writes. Use alignment padding on block_t to track this assumption. No chan...alanwu (Alan Wu)
CI runs on some platforms uJIT doesn't support. They don't have UJIT.install_entry. Maybe we want something like `install_entry_if_possible`.alanwu (Alan Wu)
* uJIT: don't compile tailcalls * Don't compile calls to protected methods We need to generate extra code to check whether the call goes through if we want to support these. * Fix copy pasta * Update blockids in branches * Update de...alanwu (Alan Wu)
Co-authored-by: Maxime Chevalier-Boisvert <maxime.chevalierboisvert@shopify.com> Co-authored-by: Alan Wu <XrXr@users.noreply.github.com>mdalessio (Mike Dalessio)
* Tie lifetime of uJIT blocks to iseqs Blocks weren't being freed when iseqs are collected. * Add rb_dary. Use it for method dependency table * Keep track of blocks per iseq Remove global version_tbl * Block version bookkeeping...alanwu (Alan Wu)
Without this fix, valgrind reports for zero-length blocks: ==149294== Invalid write of size 4 ==149294== at 0x408121: add_incoming (ujit_core.c:173) ==149294== by 0x408121: gen_block_version (ujit_core.c:286) ==149294== by 0x40...mdalessio (Mike Dalessio)
This lets us use libcapstone directly from miniruby so we don't need a Ruby Gem to to dev work. Example usage: ```ruby def foo(x) if x < 1 "wow" else "neat" end end iseq = RubyVM::InstructionSequence.of(method(:foo)) put...tenderlovemaking (Aaron Patterson)
This commit adds a module `UJIT`. The module allows you to insert the initial Micro JIT instruction in to an arbitrary iseq like this: ```ruby def foo(x) if x < 1 "less than one" else "something else" end end iseq = Ruby...tenderlovemaking (Aaron Patterson)
If the cached class uses the default allocator, then all instances coming from the class should be T_OBJECT instances. Meaning we can just check the allocator function at compile time, then skip the runtime T_OBJECT checktenderlovemaking (Aaron Patterson)
Added UJIT_CHECK_MODE. Set to 1 to double check method dispatch in generated code. It's surprising to me that we need to watch both cc and cme. There might be opportunities to simplify there.alanwu (Alan Wu)
The call cache changes when the call site becomes polymophic and can result in the generated code falsely assuming cd->cc->cme is not NULL. Here is a crasher: def body(thing) thing.strip end str = "" 10.times { b...alanwu (Alan Wu)
opt_send_without_block needs it. Ideally we would use this example only when an instruction in the compiled sequence needs `ec`. We can worry about that later.alanwu (Alan Wu)
It will read and check the inline cache now, but we need to actually call the method. Co-Authored-By: Maxime Chevalier-Boisvert <maxime.chevalierboisvert@shopify.com>tenderlovemaking (Aaron Patterson)
Insert generated addresses into st_table for mapping native code addresses back to info about VM instructions. Export `encoded_insn_data` to do this. Also some style fixes.alanwu (Alan Wu)
> I think there is only one rational answer here: Queue#freeze should prevent mutations to the Queue just like for Array/Hash/etc. I agree with this for consistency reasons. Anyone who is freezing a queue and expecting it to work has ...ioquatix (Samuel Williams)
Eregon (Benoit Daloze) wrote in #note-9: > OK, that `no benefit to changing the behavior.` was not reported here but I guess it's in the dev meeting log. ko1 (Koichi Sasada) wrote in #note-5: > ... jeremyevans0 (Jeremy Evans)
(an alternative would be to have some other way to freeze the Queue for the purpose of `deep_freeze`/`Immutable`, but I see no value to that, the way is called `freeze` and it already works for most objects).Eregon (Benoit Daloze)
OK, that `no benefit to changing the behavior.` was not reported here but I guess it's in the dev meeting log. Benefit is consistency and being able to design deep freezing/immutable without making an exception for Queue. Every time ...Eregon (Benoit Daloze)
Eregon (Benoit Daloze) wrote in #note-7: > Why was this closed? Because there was no feedback after the last message from @ko1, and it was decided at the developer meeting last year that there was no benefit to changing the behavior....jeremyevans0 (Jeremy Evans)
Why was this closed? I think there is only one rational answer here: `Queue#freeze` should prevent mutations to the Queue just like for Array/Hash/etc. There is no other "nested object" (i.e., the queue contents) exposed to user c...Eregon (Benoit Daloze)
~~~ Program received signal SIGSEGV, Segmentation fault. 0x00007ffff7d47ff8 in str_new_frozen_buffer (klass=klass@entry=4295795400, orig=orig@entry=4295625080, copy_encoding=copy_encoding@entry=1) at string.c:1329 1329 else if (STR_E...vo.x (Vit Ondruch)
My guess is that this is not really platform dependent, but instead, this is GC and it exhibits on different platforms just due to different memory layout. I think the issue is that the DTrace probe is trying to access some information w...vo.x (Vit Ondruch)
Testing Ruby with SystemTap on RHEL9 beta following these steps: ~~~ $ stap -v /usr/share/doc/ruby-doc/ruby-exercise.stp & $ ruby -e '[1, 2, 3].push(4)' ~~~ I get the following error: ~~~ /usr/share/rubygems/rubygems/errors....vo.x (Vit Ondruch)
Some notes from the discussion on Slack: In general shareable (from Ractor.make_shareable) != immutable, notably for special shareable objects like Ractor, and potentially Ractor::Array (which would share any element) or so. So the deep...Eregon (Benoit Daloze)
* [Bug #17146] Queue operations are allowed after it is frozen. (eregon) * This is an obvious inconsistency and it makes no sense whenever we discuss deep freezing object graphs. Let's fix it. Eregon (Benoit Daloze)
* [Feature #18254] Add an `offset` parameter to `String#unpack` and `String#unpack1` - Binary protocol parsers currently have to slice their buffer repeatedly, this new argument would save a lot of string copying. byroot (Jean Boussier)
* [Feature #17795] Around `Process.fork` callbacks API (dan0042) * A few naming suggestions have been made; does Matz like any of them? * [Feature #17837] Add support for Regexp timeouts (dan0042) * Adding this to ruby 3.1 with a ...Dan0042 (Daniel DeLorme)
nobu (Nobuyoshi Nakada) wrote in #note-3: > Found the bug. > ... Standby, I'm still setting up the environment to build for the embedded machine. vihai (Daniele Orlandi)
> In the description's code, there is lock and unlock. Are those supposed to be thread-safe? If yes I think you'd need to synchronize in almost every method, if it's possible to access the buffer without GVL. No, instances of this class...ioquatix (Samuel Williams)
Thanks. I have another PR for the same issue in https://github.com/ruby/error_highlight/pull/11 I'm okay for the change, but I'd like @ko1 to review the change.mame (Yusuke Endoh)
I looked into this. If we removed `RUBY_DEVEL`, we would make it so `ruby_debug_log` was never defined, since it is only defined if `RUBY_DEVEL` is defined. Basically, removing `RUBY_DEVEL` is equivalent to removing the removing the en...jeremyevans0 (Jeremy Evans)
ko1 (Koichi Sasada) wrote in #note-18: > Another proposal is change `$!` semantics to `begin/rescue/ensure' clause local. > ... That seems elegant actually, because ```ruby begin ... rescue foo($!) end ``` would behave lik...Eregon (Benoit Daloze)
~~Found the bug. Does this patch fix it?~~ ```diff diff --git a/io.c b/io.c index 50c9fea62c9..052155205d6 100644 --- a/io.c +++ b/io.c @@ -10143,8 +10143,8 @@ setup_narg(ioctl_req_t cmd, VALUE *argp, int io_p) /* expand for data ...nobu (Nobuyoshi Nakada)
nobu (Nobuyoshi Nakada) wrote in #note-1: > > ... "X"*364 is a request that makes ioctl fail and the SystemCallError is raised before the sanity check of the buffer. However, given `req` as a proper request as a binary string: ``` > f...vihai (Daniele Orlandi)
vihai (Daniele Orlandi) wrote: > The ioctl sanity check is triggered as if the buffer was too small however the size of the buffer passed to ioctl is correct. > ... Do you mean that `fd.ioctl(GPIO_GET_LINEHANDLE_IOCTL, "X"*364)` raises...nobu (Nobuyoshi Nakada)
nobu (Nobuyoshi Nakada) wrote in #note-3: > The current names have been moved from the top level, to make clarify the roles. I don't understand this reasoning. What else could a Mutex/Queue/SizedQueue/ConditionVariable be for if for ...Eregon (Benoit Daloze)
Related, there is this PR by nobu to make ruby/spec use the `Thread::` prefix: https://github.com/ruby/spec/pull/867 I agree it makes sense to align the canonical name and specs hierarchy (although I think it's not too important either)...Eregon (Benoit Daloze)
Currently these 4 classes are defined as both constants of `Object` and of `Thread`. On CRuby 3.0.2, their `#inspect` shows they were first defined under `Thread` and then aliased in Object: ``` $ ruby -ve 'p [Mutex, Queue, SizedQue...Eregon (Benoit Daloze)
In the description's code, there is `lock` and `unlock`. Are those supposed to be thread-safe? If yes I think you'd need to synchronize in almost every method, if it's possible to access the buffer without GVL. I think it's better to o...Eregon (Benoit Daloze)
> mame: I cannot understand what is finally needed. Doesn’t String with ASCII-8BIT work? String is both insufficient and inefficient. You can check how read and write on strings work with internal frozen copies, for example, it's both p...ioquatix (Samuel Williams)
Today we read the ticket (not all comments, sorry) and mame, ko1 has comment: mame: I cannot understand what is finally needed. Doesn’t String with ASCII-8BIT work? ko1: I can’t understand how to use it with IO? Only for scheduler?ko1 (Koichi Sasada)
* [Feature #18256] Change the canonical name of Thread::Mutex, Thread::Queue, Thread::SizedQueue and Thread::ConditionVariable to just Mutex, Queue, SizedQueue and ConditionVariable (eregon) * OK? (sorry for late addition for the meet...Eregon (Benoit Daloze)
ioquatix (Samuel Williams) wrote in #note-17: > @ko1 changed `Process::Status` to be frozen by default. What is the logic? What is the problem we are trying to solve by doing this? Is it to make things sharable by Ractor? Yes.ko1 (Koichi Sasada)
Hello, I'm running ruby 2.7.4p191 on an armv7 linux and experimenting with GPIO_GET_LINEHANDLE_IOCTL ioctl. The ioctl sanity check is triggered as if the buffer was too small however the size of the buffer passed to ioctl is corr...vihai (Daniele Orlandi)
I think this is a useful feature, and it can be implemented without any API changes/backwards compatibility issues. I submitted a pull request for it: https://github.com/ruby/ruby/pull/4986jeremyevans0 (Jeremy Evans)
Ah, I didn't know about it, but then you just allocated a string and converted an integer to string, so it's even slower than the `slice` pattern: ```ruby # frozen_string_literal: true require 'benchmark/ips' STRING = Random.byte...byroot (Jean Boussier)
When working with binary protocols it's common to have to first unpack some kind of header or type prefix, and then based on that unpack another part of the string. For instance here's [a code snippet from Dalli, the most common Memca...byroot (Jean Boussier)
The `newhash` instruction was checking if `num` is greater than 0, but so is [`rb_hash_new_with_size`](https://github.com/ruby/ruby/blob/82e2443d8b1e3edd2607c78dddf5aac79a13492d/hash.c#L1564) as well as [`rb_hash_bulk_insert`](https://gi...tenderlovemaking (Aaron Patterson)
`rb_id_table_foreach_with_replace` doesn't pass the `ID` to `func` and `replace`. If this is intensional, `func` should be `rb_id_table_foreach_values_func_t` and `rb_id_table_update_callback_func_t` doesn't need the `id` argument? ...nobu (Nobuyoshi Nakada)
Here's hoping that a name, any name, will get matz' approval so this can make it into ruby 3.1 `Process._fork` A name begining with an underscore is often used to indicate an internal/private method. `Process.fork!` A bang to com...Dan0042 (Daniel DeLorme)
So if we have 536870911 backtracks per 48.6898231/10 seconds, that comes out to roughly 110M backtracks per second. How about fixing a safe limit of 60s -> 6600M backtracks? Since it only stops the most pathological regexp after 60s, tha...Dan0042 (Daniel DeLorme)
Drop support for Ruby 2.3, 2.4, and 2.5. As of 2021-10, Ruby 2.6 is the oldest version that still receives security fixes from the Ruby core team, so it doesn't make much sense to keep code for those ancient versions. https://github.co...rhenium (Kazuki Yamaguchi)
LibreSSL 2.2.x has a bug in the Finished message handling with TLS 1.3. This is fixed by LibreSSL 3.3.2. https://github.com/ruby/openssl/commit/0bea59d245rhenium (Kazuki Yamaguchi)
On the server side, the serialized list of protocols is stored in SSL_CTX as a String object reference. We utilize a hidden instance variable to prevent it from being GC'ed, but this is not enough because it can also be relocated by GC.c...rhenium (Kazuki Yamaguchi)
We store the reverse reference to the Ruby object in the OpenSSL struct for use from OpenSSL callback functions. To prevent the Ruby object from being relocated by GC.compact, we must "pin" it by calling rb_gc_mark(). https://github.com...rhenium (Kazuki Yamaguchi)
We store the reverse reference to the Ruby object in the OpenSSL struct for use from OpenSSL callback functions. To prevent the Ruby object from being relocated by GC.compact, we must "pin" it by calling rb_gc_mark(). https://github.com...rhenium (Kazuki Yamaguchi)
The digest library is a default gem now, too. Therefore we can't simply use rb_require() to load it, but we should use Kernel#require instead. This change is based on the suggestion by David Rodríguez in https://github.com/ruby/digest/c...rhenium (Kazuki Yamaguchi)
Similarly to SSLSocket#syswrite, the blocking SSLSocket#sysread allows context switches. We must prevent other threads from modifying the string buffer. We can use rb_str_locktmp() and rb_str_unlocktmp() to temporarily prohibit modifica...rhenium (Kazuki Yamaguchi)
Since a blocking SSLSocket#syswrite call allows context switches while waiting for the underlying socket to be ready, we must freeze the string buffer to prevent other threads from modifying it. Reference: https://github.com/ruby/openss...rhenium (Kazuki Yamaguchi)
[ This is a backport to the 2.1 branch. ] Fixed misspellings reported at [Bug #16437], for default gems. (cherry picked from commit ruby/ruby@e68999c82c4863d33a6f893661fba1b7538c5671) https://github.com/ruby/openssl/commit/0f43789503nobu (Nobuyoshi Nakada)
Provide a wrapper of SSL_set0_tmp_dh_pkey()/SSL_CTX_set_tmp_dh(), which sets the DH parameters used for ephemeral DH key exchange. SSLContext#tmp_dh_callback= already exists for this purpose, as a wrapper around SSL_CTX_set_tmp_dh_callb...rhenium (Kazuki Yamaguchi)
openssl gem version 2.1.3 and 2.2.1 have been released with this change in addition to other bug fixes/improvements. Backport to ruby core is requested in [Bug #18251] and [Bug #18252].rhenium (Kazuki Yamaguchi)
Please merge openssl 2.2.1 into ruby_3_0. ruby_3_0's openssl is currently at version 2.2.0 with local modifications. openssl 2.2.1 incorporated these and includes some additional bug fixes: https://github.com/ruby/openssl/compare/v2.2...rhenium (Kazuki Yamaguchi)
Please merge openssl 2.1.3 into ruby_2_7. ruby_2_7's openssl is currently at version 2.1.2 with local modifications. openssl 2.1.3 incorporated these and includes some additional bug fixes: https://github.com/ruby/openssl/compare/v2.1...rhenium (Kazuki Yamaguchi)
Fix Ractor.make_shareable changing locals for Procs env_copy() uses rb_ary_delete_at() with a loop counting up while iterating through the list of read only locals. rb_ary_delete_at() can shift elements in the ar...nagachika (Tomoyuki Chikanaga)
Dump outer variables tables when dumping an iseq to binary This commit dumps the outer variables table when dumping an iseq to binary. This fixes a case where Ractors aren't able to tell what outer variables bel...nagachika (Tomoyuki Chikanaga)
I updated https://bugs.ruby-lang.org/projects/ruby/wiki/HowToContribute to discuss testing and where to document changes. The guide already contained a statement to follow the style of the existing code, and I don't think we need to be ...jeremyevans0 (Jeremy Evans)
This change fixes alias call-seq to return nil if the method's call-seq does not specify the alias. Previously, the alias's call-seq would be an empty string in this case which broke darkfish rendering. This change also backfills test ...mdalessio (Mike Dalessio)
Applied in changeset commit:git|2a5c3a4d0f693ad0fe7b76dd99155e57149d2cac. ---------- Update documentation for String and Symbol to discuss differences Implements [Feature #14347]jeremyevans (Jeremy Evans)
I'd like this feature to make 3.1, so I submitted a pull request for it, borrowing somewhat from @fatkodima's pull request: https://github.com/ruby/ruby/pull/4974 As @ko1 specified, the receiver is not included in the array, nor are s...jeremyevans0 (Jeremy Evans)
> then it should bump the major version (or some other well-documented scheme). I can hardly see something like this working in practice. Maybe for digest it would work as realistically it won't change much in the future. But if Rubyg...byroot (Jean Boussier)
byroot (Jean Boussier) wrote in #note-8: > e.g. I have `app1` with `digest 3.0.0` and `app2` with `digest 3.2.0`, both have `somegem-that-link-to-digest`, this means we'd need to compile that gem twice and link against different headers...Eregon (Benoit Daloze)
byroot (Jean Boussier) wrote in #note-8: > But then I suppose them that compile against `digest.h` would also need to know where to look, I'm really unsure how it's supposed to work. If I have 5 different versions of digest, which one s...Eregon (Benoit Daloze)
Kind of yes. From my understanding the problem is that $INSTALL_FILES always install files in `site_ruby` https://github.com/ruby/digest/blob/1cb6bac1a6732a1415278b94fad2fdc695477dec/ext/digest/extconf.rb#L7-L9 So from my limited unde...byroot (Jean Boussier)
Hi, no I wasn't, and I'm not sure how it should be fixed. If I understand correctly, this is about cases where rubygems does not have write access to the location where ruby itself is installed, while it does have write access to the loc...deivid (David Rodríguez)
@deivid are you aware of this issue? Any idea how we could fix this or what decision would need to be taken? I fear that the `digest` extraction might cause major problem to people trying to upgrade to 3.1 next year, or trying to test...byroot (Jean Boussier)
See related commits: - ebc66662 for #=== - 4943d208 for #[], #[]=, #<<, and #>> https://github.com/ruby/rdoc/commit/8e47f7840amdalessio (Mike Dalessio)
Previously, Parser::C comments all defaulted to "rdoc" format, even when the user had set a different default with the `--markup=<choice>` option. https://github.com/ruby/rdoc/commit/4643b08a26mdalessio (Mike Dalessio)
@jeremyevans added a test for this in https://github.com/ruby/ruby/commit/2188d6d160d3ba82432c87277310a4d417e136d5#diff-8b146f8f5073c5ee6526a128c3e9c05ae7a52d2cd496f601351470137f2a87a0 but it was removed by @nobu in https://github.com/r...Eregon (Benoit Daloze)
- [Feature #17760] Where we should install a header file when `gem install --user`? - `digest` have been gemified. However its `extconf.rb` have `$INSTALLFILES = { "digest.h" => "$(HDRDIR)" }` which fail for most people because ruby's ...byroot (Jean Boussier)
I figured the `digest` issue. It only happens if you have a `git` source gem as well in the Gemfile, because the git source does a MD5 of the repository URL. I submitted a PR that could fix the problem: https://github.com/rubygems/rub...byroot (Jean Boussier)
See also: https://github.com/ruby/csv/issues/117#issuecomment-933289373 How to reproduce with x.csv.gz in the issue comment: Zlib::GzipReader.open("x.csv.gz") do |rio| rio.gets(nil, 1024) while line = rio.gets(nil, 8192...Sutou Kouhei
@jeremyevans0 Thanks. Everything seems fine with master. I looked at a commit somewhat later than the version listed (2020-06-11), and all the bin stubs were contained in the `bin` folder. Today, they are generated in another manner;...MSP-Greg (Greg L)
11:06 PM Feature #11066 (Rejected): symbol.to_s performance - caching and returning a frozen string
10:47 PM Revision 844588f9 (git): Push compaction page alignment check down
08:34 PM Bug #18260: YJIT uses insecure mmap protections
07:07 PM Revision bdfc23cb (git): YJIT: don't compile attr_accessor methods when tracing (#4998)
06:33 PM Revision d0cad8ad (git): [rubygems/rubygems] Re-enable `default_ignores` option for standard
03:26 PM Revision 119626da (git): Force disable yjit on OpenBSD
08:11 AM Bug #18187: Float#clamp() returns ArgumentError (comparison of Float with 1 failed)
01:57 PM Bug #18257: rb_mRubyVMFrozenCore is broken by GC run
01:34 PM Bug #18255: ioctl zeroes the last buffer byte
11:17 AM Feature #18083: Capture error in ensure block.
10:24 AM Feature #18083: Capture error in ensure block.
06:13 AM Feature #18083: Capture error in ensure block.
12:31 PM Feature #17873: Update of default gems in Ruby 3.1
12:15 PM Feature #18259: Support quarter spec %q in Time#strftime
09:39 AM Feature #18259 (Open): Support quarter spec %q in Time#strftime
11:37 AM Misc #18174: DevelopersMeeting20211021Japan
09:41 AM Feature #17760: Where we should install a header file when `gem install --user`?
07:36 AM Bug #17429 (Closed): Prohibit include/prepend in refinement modules
05:08 AM Revision 3b16d07e (git): Fix test_version again for RUBY_YJIT_ENABLE=1
12:25 AM Feature #18256: Change the canonical name of Thread::Mutex, Thread::Queue, Thread::SizedQueue and Thread::ConditionVariable to just Mutex, Queue, SizedQueue and ConditionVariable
10:19 PM Revision 4cad8930 (git): Add String#bytesize
10:19 PM Revision 13261f00 (git): More simple bootstrap tests for kwargs
10:19 PM Revision 5b68d14c (git): Add optimized Thread.current
10:19 PM Revision 217f7cb1 (git): Include errno message in mmap failure output
10:19 PM Revision 918668b4 (git): update README with correct repository URL
04:02 PM Bug #18257: rb_mRubyVMFrozenCore is broken by GC run
01:28 AM Misc #18174: DevelopersMeeting20211021Japan
05:00 PM Revision 3e96b94e (git): Enhanced RDoc for Numeric (#4991)
04:48 AM Bug #18187: Float#clamp() returns ArgumentError (comparison of Float with 1 failed)
10:39 AM Revision cc8ff8b5 (git): [ruby/openssl] require Ruby 2.6 or later
09:34 AM Revision f88401f3 (git): [ruby/openssl] fix segv in Timestamp::{Request,Response,TokenInfo}.new
04:56 AM Bug #18232: Ractor.make_shareable is broken in code loaded with RubyVM::InstructionSequence.load_from_binary
08:05 AM Feature #17760: Where we should install a header file when `gem install --user`?
01:57 AM Bug #16953: Windows bash bin files removed in master