Fix CRLF -> LF conversion on read for rb_io_fdopen & rb_file_open
When opening a file with File.open, and then setting the encoding with IO#set_encoding, it still correctly performs CRLF -> LF conversion on Windows when reading files with a CRLF line ending in them (in text mode).
However, the file is opened instead with either the rb_io_fdopen or rb_file_open APIs from C, the CRLF conversion is NOT set up correctly; it works if the encoding is not specified, but if IO#set_encoding is called, the conversion stops happening. This seems to be because the encflags never get ECONV_DEFAULT_NEWLINE_DECORATOR set in these codepaths.
Concretely, this means that the conversion doesn't happen in the following circumstances:
When loading ruby files with require (that calls rb_io_fdopen)
When parsing ruuby files with RubyVM::AbstractSyntaxTree (that calls rb_file_open). This then causes the ErrorHighlight tests to fail on windows if git has checked them out with CRLF line endings - the error messages it's testing wind up with literal \r\n sequences in them because the iseq text from the parser contains un-newline-converted strings.
This commit fixes the problem by copy-pasting the relevant snippet which sets this up in rb_io_extract_modeenc (for the File.open path) into the relevant codepaths for rb_io_fdopen and rb_file_open.
Fix CRLF -> LF conversion on read for rb_io_fdopen & rb_file_open
When opening a file with
File.open, and then setting the encoding withIO#set_encoding, it still correctly performs CRLF -> LF conversion onWindows when reading files with a CRLF line ending in them (in text
mode).
However, the file is opened instead with either the
rb_io_fdopenorrb_file_openAPIs from C, the CRLF conversion is NOT set upcorrectly; it works if the encoding is not specified, but if
IO#set_encodingis called, the conversion stops happening. This seemsto be because the encflags never get ECONV_DEFAULT_NEWLINE_DECORATOR
set in these codepaths.
Concretely, this means that the conversion doesn't happen in the
following circumstances:
rb_file_open).
This then causes the ErrorHighlight tests to fail on windows if git has
checked them out with CRLF line endings - the error messages it's
testing wind up with literal \r\n sequences in them because the iseq
text from the parser contains un-newline-converted strings.
This commit fixes the problem by copy-pasting the relevant snippet which
sets this up in
rb_io_extract_modeenc(for the File.open path) intothe relevant codepaths for
rb_io_fdopenandrb_file_open.[Bug #20101]