In #75528, we would like to tweak the structure of the tokens rustc uses. One of the components which heavily depends on the current structure is rustdoc (html and check_code_block_syntax module use lexer::StringReader).

I think, for this issue and in general, it makes sense to switch rustdoc over to rustc_lexer, which produces lower-level guaranteed lossless tokens, which look like a better fit for the task of syntax highlighting.

I plan to do this refactor myself, but I want to check with @rust-lang/rustdoc if this is a good idea.

EDIT: cc #63284