Actions
Bug #18962
closed
Ripper.lex does not handle \0, ^D and ^Z correctly.
Bug #18962: Ripper.lex does not handle \0, ^D and ^Z correctly.
Status:
Closed
Assignee:
-
Target version:
-
ruby -v:
ruby 3.1.0p0 (2021-12-25 revision fb4df44d16) [x86_64-darwin20]
Description
Ripper.lex does not end with \0, ^D and ^Z, and it contains weird tokens including "\u0000"
# for syntax ok code Ripper.lex("hello\0\0world").map{_3} #=> ["hello"] Ripper.lex("hello\0world").map{_3} #=> ["hello", "\u0000world"] Ripper.lex("hello\0@world").map{_3} #=> ["hello", "@world"] Ripper.lex("hello\0:world").map{_3} #=> ["hello", "\u0000:", "world"] Ripper.lex("hello\0))").map{_3} #=> ["hello", "\u0000)", ")"] # for syntax invalid code Ripper.lex("end\0*").map{_3} # => ["end", "\u0000*"] Ripper.lex("end\0\0*").map{_3} # => ["end"] Ripper.lex("end\0@end").map{_3} # => ["end", "@end"] 
Updated by nobu (Nobuyoshi Nakada) about 3 years ago
- Status changed from Open to Closed
Applied in changeset git|844a0edbae6e74293e3d0cb1ceeeb66a4371d06d.
[Bug #18962] Do not read again once reached EOF
Ripper::Lexer#parse re-parses the source code with syntax errors
when raise_errors: false.
Co-Authored-By: tompng tomoyapenguin@gmail.com
Actions