Enable ruby2_keywords semantics by default

[eregon]

See https://bugs.ruby-lang.org/issues/16463
Slides: https://docs.google.com/presentation/d/1J6voqHFQ46-MsEm_vUJsBJiktNvoA6niz3fea9awmco/edit?usp=sharing

[eregon]

@mame @jeremyevans This passes test-spec and test-all, and it's a single line change in parse.y.
Please review commit-by-commit as the commit messages explain in details each change.

[eregon]

Of course with this change, calling ruby2_keywords no longer changes anything, so we can remove those calls from the code in this repository (which cleans up the code).

It's not in this PR (yet) to keep it minimal and easy to review. I'm happy to add a commit doing that though. We might possibly also want a $VERBOSE==true warning saying ruby2_keywords no longer has any effect.

[jeremyevans]

Implementation-wise, this looks like the best way to make this change, and should make backporting the feature easy. I'm still not in favor of the change, but if matz approves it, this is definitely the best way to implement it.

[eregon]

Here is the same PR based on ruby_2_7: ruby_2_7...eregon:ruby2_keywords-by-default-27
It's the same commits, just rebased on that branch.

[eregon]

And here we go, the proof that the entire Rails test suite is green with this change on top of 2.7.0!
(the run with my branch is labeled master)

rails/rails#38299
https://buildkite.com/rails/rails/builds/66631#_

In fact, it's even more compatible with 2.6, and does not need RUBY_VERSION < "2.7" checks in tests.

And as we can see in https://github.com/rails/rails/blob/69a99e47c063f5d238e6d68f999f44f063aaf478/railties/test/generators/actions_test.rb#L576-L584 people are tempted to use *args, **kwargs delegation in 2.7 but that's broken in 2.7 (& in 2.6 too), so we should only warn delegation (= tell users to migrate) when *args, **kwargs delegation works correctly, in Ruby 3.0+.

[eregon]

Also, this branch would allow a nice cleanup to remove 42 calls to ruby2_keywords in the Rails codebase, if 2.7.0 doesn't need to be supported (2.7.0 already requires a pretty ugly hack for Hash.ruby2_keywords_hash?).
Those ruby2_keywords look like:

ruby2_keywords(:use) if respond_to?(:ruby2_keywords, true)

[eregon]

Also worth noting that this PR based on ruby master seems more compatible than ruby master for Rails tests. I would guess because *args-delegation keeps working and not every library is migrated yet (also considering it's not easy to migrate delegation with 2.7).

Results for ruby master: https://buildkite.com/rails/rails/builds/66628
Results for this branch: https://buildkite.com/rails/rails/builds/66634
activestorage tests fail with ruby master (ArgumentError: wrong number of arguments (given 4, expected 2..3)), but pass with this branch.

[eregon]

I rebased this PR. With the recent changes by @jeremyevans to remove **empty_hash even for ruby2_keywords, this diff is becoming even nicer:

• No spec is changed.
• No test is changed, except one where clearly this PR behavior is better.
• I added new tests for *args-delegation.

[eregon]

Matz said no to this, so I'll close this PR.

Maybe it could become a disabled-by-default flag to help transition, on MRI or other Ruby implementations if there is interest.

It seems like the general direction is to force migrating delegation by 3.0 and to have to use ruby2_keywords. I guess many might skip 2.7 since the delegation warnings are so confusing and so hard to address and in 3.0 it's significantly easier to address delegation in existing codebases as the semantics are better defined. OTOH, non-delegation warnings are good in 2.7 and break hard in 3.0, so there is no good version to migrate all keyword-arguments-related logic.

I feel sad that we have to use an explicit ruby2_keywords and migrate delegation code twice with little help (and even marking all these methods as ruby2_keywords won't be enough for the second migration later once the ruby2_keywords hack is removed), but that's how it is.