[rustdoc] Correctly handle should_panic doctest attribute and fix --no-run test flag on the 2024 edition

[GuillaumeGomez]

Fixes #143009.
Fixes #143858.

Supersedes #143900.

For --no-run, we forgot to check the "global" options in the 2024 edition, fixed in the first commit.

For should_panic fix, the exit code check has been fixed.

r? @fmease

[rustbot]

fmease is currently at their maximum review capacity.
They may take a while to respond.

[GuillaumeGomez]

Let's see if it works like that, otherwise I'll add some cfg(bootstrap) around. Locally with stage 2 it worked fine.

[GuillaumeGomez]

Ah right, forgot to fix std doc examples.

[GuillaumeGomez]

Checking all previously failing CIs.

@bors try jobs=armhf-gnu,test-various,x86_64-gnu-aux

[rust-bors]

💔 Test for 0c093af failed: CI. Failed jobs:

• try - test-various (web logs, extended logs)

[GuillaumeGomez]

So the error comes from the fact that should_panic is ignored on some platforms by libtest and rustdoc didn't do this check. I added it as well.

Let's try again.

@bors try jobs=armhf-gnu,test-various,x86_64-gnu-aux

[rust-bors]

💔 Test for 7103c01 failed: CI. Failed jobs:

• try - test-various (web logs, extended logs)

[GuillaumeGomez]

@bors try jobs=armhf-gnu,test-various,x86_64-gnu-aux

[rust-bors]

💔 Test for 38c3e03 failed: CI. Failed jobs:

• try - test-various (web logs, extended logs)

[GuillaumeGomez]

@bors try jobs=test-various

[rust-bors]

💔 Test for c5583f8 failed: CI. Failed jobs:

• try - test-various (web logs, extended logs)

[GuillaumeGomez]

Not ready yet! Just more cleanups before my next attempt: if should_panic (and not compile_fail), then we compile a binary with libtest to ensure we can actually run this test and if so, we call the other code.

[GuillaumeGomez]

@bors try jobs=test-various

[rust-log-analyzer]

The job aarch64-gnu-llvm-20-2 failed! Check out the build log: (web) (plain enhanced) (plain)

Click to see the possible cause of the failure (guessed by this bot)

---- doctest::tests::comment_in_attrs stdout ---- thread 'doctest::tests::comment_in_attrs' (56156) panicked at src/librustdoc/doctest/tests.rs:419:5: assertion `left == right` failed left: ("#![allow(unused)]\n#![feature(rustdoc_internals)]\n#![allow(internal_features)]\n#![doc(rust_logo)]\n//! This crate has the Rust(tm) branding on it.\npub fn main() {\n\n}", 2) right: ("#![allow(unused)]\n#![feature(rustdoc_internals)]\n#![allow(internal_features)]\n#![doc(rust_logo)]\n//! This crate has the Rust(tm) branding on it.\nfn main() {\n\n}", 2) stack backtrace: 0: __rustc::rust_begin_unwind 1: core::panicking::panic_fmt 2: core::panicking::assert_failed_inner 3: core::panicking::assert_failed::<(alloc::string::String, usize), (alloc::string::String, usize)> 4: rustdoc::doctest::tests::comment_in_attrs 5: <rustdoc::doctest::tests::comment_in_attrs::{closure#0} as core::ops::function::FnOnce<()>>::call_once note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace. ---- doctest::tests::comment_in_attrs stdout end ---- ---- doctest::tests::make_test_basic stdout ---- thread 'doctest::tests::make_test_basic' (56158) panicked at src/librustdoc/doctest/tests.rs:53:5: assertion `left == right` failed left: ("#![allow(unused)]\npub fn main() {\nassert_eq!(2+2, 4);\n}", 2) right: ("#![allow(unused)]\nfn main() {\nassert_eq!(2+2, 4);\n}", 2) stack backtrace: 0: __rustc::rust_begin_unwind 1: core::panicking::panic_fmt 2: core::panicking::assert_failed_inner 3: core::panicking::assert_failed::<(alloc::string::String, usize), (alloc::string::String, usize)> 4: rustdoc::doctest::tests::make_test_basic 5: <rustdoc::doctest::tests::make_test_basic::{closure#0} as core::ops::function::FnOnce<()>>::call_once note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace. ---- doctest::tests::make_test_basic stdout end ---- ---- doctest::tests::inner_attributes stdout ---- thread 'doctest::tests::inner_attributes' (56157) panicked at src/librustdoc/doctest/tests.rs:467:5: assertion `left == right` failed left: ("#![allow(unused)]\n\n//! A doc comment that applies to the implicit anonymous module of this crate\n\n\npub fn main() {\npub mod outer_module {\n //!! - Still an inner line doc (but with a bang at the beginning)\n}\n}", 2) right: ("#![allow(unused)]\n\n//! A doc comment that applies to the implicit anonymous module of this crate\n\n\nfn main() {\npub mod outer_module {\n //!! - Still an inner line doc (but with a bang at the beginning)\n}\n}", 2) stack backtrace: 0: __rustc::rust_begin_unwind 1: core::panicking::panic_fmt 2: core::panicking::assert_failed_inner 3: core::panicking::assert_failed::<(alloc::string::String, usize), (alloc::string::String, usize)> 4: rustdoc::doctest::tests::inner_attributes 5: <rustdoc::doctest::tests::inner_attributes::{closure#0} as core::ops::function::FnOnce<()>>::call_once note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace. ---- doctest::tests::inner_attributes stdout end ---- ---- doctest::tests::make_test_crate_attrs stdout ---- thread 'doctest::tests::make_test_crate_attrs' (56159) panicked at src/librustdoc/doctest/tests.rs:219:5: assertion `left == right` failed left: ("#![allow(unused)]\n#![feature(sick_rad)]\n\npub fn main() {\nassert_eq!(2+2, 4);\n}", 2) right: ("#![allow(unused)]\n#![feature(sick_rad)]\n\nfn main() {\nassert_eq!(2+2, 4);\n}", 2) stack backtrace: 0: __rustc::rust_begin_unwind 1: core::panicking::panic_fmt 2: core::panicking::assert_failed_inner 3: core::panicking::assert_failed::<(alloc::string::String, usize), (alloc::string::String, usize)> 4: rustdoc::doctest::tests::make_test_crate_attrs 5: <rustdoc::doctest::tests::make_test_crate_attrs::{closure#0} as core::ops::function::FnOnce<()>>::call_once note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace. ---- doctest::tests::make_test_crate_attrs stdout end ---- ---- doctest::tests::make_test_crate_name stdout ---- thread 'doctest::tests::make_test_crate_name' (56160) panicked at src/librustdoc/doctest/tests.rs:87:5: assertion `left == right` failed left: ("#![allow(unused)]\n#[allow(unused_extern_crates)]\nextern crate r#asdf;\npub fn main() {\nuse asdf::qwop;\nassert_eq!(2+2, 4);\n}", 3) right: ("#![allow(unused)]\n#[allow(unused_extern_crates)]\nextern crate r#asdf;\nfn main() {\nuse asdf::qwop;\nassert_eq!(2+2, 4);\n}", 3) stack backtrace: 0: __rustc::rust_begin_unwind 1: core::panicking::panic_fmt 2: core::panicking::assert_failed_inner 3: core::panicking::assert_failed::<(alloc::string::String, usize), (alloc::string::String, usize)> 4: rustdoc::doctest::tests::make_test_crate_name 5: <rustdoc::doctest::tests::make_test_crate_name::{closure#0} as core::ops::function::FnOnce<()>>::call_once note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace. ---- doctest::tests::make_test_crate_name stdout end ---- ---- doctest::tests::make_test_fake_main stdout ---- thread 'doctest::tests::make_test_fake_main' (56163) panicked at src/librustdoc/doctest/tests.rs:251:5: assertion `left == right` failed left: ("#![allow(unused)]\npub fn main() {\n//Ceci n'est pas une `fn main`\nassert_eq!(2+2, 4);\n}", 2) right: ("#![allow(unused)]\nfn main() {\n//Ceci n'est pas une `fn main`\nassert_eq!(2+2, 4);\n}", 2) stack backtrace: 0: __rustc::rust_begin_unwind 1: core::panicking::panic_fmt 2: core::panicking::assert_failed_inner 3: core::panicking::assert_failed::<(alloc::string::String, usize), (alloc::string::String, usize)> 4: rustdoc::doctest::tests::make_test_fake_main 5: <rustdoc::doctest::tests::make_test_fake_main::{closure#0} as core::ops::function::FnOnce<()>>::call_once note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace. ---- doctest::tests::make_test_fake_main stdout end ---- ---- doctest::tests::make_test_crate_name_no_use stdout ---- thread 'doctest::tests::make_test_crate_name_no_use' (56161) panicked at src/librustdoc/doctest/tests.rs:68:5: assertion `left == right` failed left: ("#![allow(unused)]\npub fn main() {\nassert_eq!(2+2, 4);\n}", 2) right: ("#![allow(unused)]\nfn main() {\nassert_eq!(2+2, 4);\n}", 2) stack backtrace: 0: __rustc::rust_begin_unwind 1: core::panicking::panic_fmt 2: core::panicking::assert_failed_inner 3: core::panicking::assert_failed::<(alloc::string::String, usize), (alloc::string::String, usize)> 4: rustdoc::doctest::tests::make_test_crate_name_no_use 5: <rustdoc::doctest::tests::make_test_crate_name_no_use::{closure#0} as core::ops::function::FnOnce<()>>::call_once note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace. ---- doctest::tests::make_test_crate_name_no_use stdout end ---- ---- doctest::tests::make_test_ignore_std stdout ---- thread 'doctest::tests::make_test_ignore_std' (56164) panicked at src/librustdoc/doctest/tests.rs:122:5: assertion `left == right` failed left: ("#![allow(unused)]\npub fn main() {\nuse std::*;\nassert_eq!(2+2, 4);\n}", 2) right: ("#![allow(unused)]\nfn main() {\nuse std::*;\nassert_eq!(2+2, 4);\n}", 2) stack backtrace: 0: __rustc::rust_begin_unwind 1: core::panicking::panic_fmt 2: core::panicking::assert_failed_inner 3: core::panicking::assert_failed::<(alloc::string::String, usize), (alloc::string::String, usize)> 4: rustdoc::doctest::tests::make_test_ignore_std 5: <rustdoc::doctest::tests::make_test_ignore_std::{closure#0} as core::ops::function::FnOnce<()>>::call_once note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace. ---- doctest::tests::make_test_ignore_std stdout end ---- ---- doctest::tests::make_test_insert_extra_space stdout ---- thread 'doctest::tests::make_test_insert_extra_space' (56165) panicked at src/librustdoc/doctest/tests.rs:374:5: assertion `left == right` failed left: ("#![allow(unused)]\npub fn main() {\n use std::*;\n assert_eq!(2+2, 4);\n eprintln!(\"hello anan\");\n}", 2) right: ("#![allow(unused)]\nfn main() {\n use std::*;\n assert_eq!(2+2, 4);\n eprintln!(\"hello anan\");\n}", 2) stack backtrace: 0: __rustc::rust_begin_unwind 1: core::panicking::panic_fmt 2: core::panicking::assert_failed_inner 3: core::panicking::assert_failed::<(alloc::string::String, usize), (alloc::string::String, usize)> 4: rustdoc::doctest::tests::make_test_insert_extra_space 5: <rustdoc::doctest::tests::make_test_insert_extra_space::{closure#0} as core::ops::function::FnOnce<()>>::call_once note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace. ---- doctest::tests::make_test_insert_extra_space stdout end ---- ---- doctest::tests::make_test_issues_33731 stdout ---- thread 'doctest::tests::make_test_issues_33731' (56168) panicked at src/librustdoc/doctest/tests.rs:303:5: assertion `left == right` failed left: ("#![allow(unused)]\nextern crate hella_qwop;\n#[allow(unused_extern_crates)]\nextern crate r#asdf;\npub fn main() {\nassert_eq!(asdf::foo, 4);\n}", 3) right: ("#![allow(unused)]\nextern crate hella_qwop;\n#[allow(unused_extern_crates)]\nextern crate r#asdf;\nfn main() {\nassert_eq!(asdf::foo, 4);\n}", 3) stack backtrace: 0: __rustc::rust_begin_unwind 1: core::panicking::panic_fmt 2: core::panicking::assert_failed_inner 3: core::panicking::assert_failed::<(alloc::string::String, usize), (alloc::string::String, usize)> 4: rustdoc::doctest::tests::make_test_issues_33731 5: <rustdoc::doctest::tests::make_test_issues_33731::{closure#0} as core::ops::function::FnOnce<()>>::call_once note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace. ---- doctest::tests::make_test_issues_33731 stdout end ---- ---- doctest::tests::make_test_manual_extern_crate stdout ---- thread 'doctest::tests::make_test_manual_extern_crate' (56170) panicked at src/librustdoc/doctest/tests.rs:141:5: assertion `left == right` failed left: ("#![allow(unused)]\nextern crate asdf;\npub fn main() {\nuse asdf::qwop;\nassert_eq!(2+2, 4);\n}", 2) right: ("#![allow(unused)]\nextern crate asdf;\nfn main() {\nuse asdf::qwop;\nassert_eq!(2+2, 4);\n}", 2) stack backtrace: 0: __rustc::rust_begin_unwind 1: core::panicking::panic_fmt 2: core::panicking::assert_failed_inner 3: core::panicking::assert_failed::<(alloc::string::String, usize), (alloc::string::String, usize)> 4: rustdoc::doctest::tests::make_test_manual_extern_crate 5: <rustdoc::doctest::tests::make_test_manual_extern_crate::{closure#0} as core::ops::function::FnOnce<()>>::call_once note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace. ---- doctest::tests::make_test_manual_extern_crate stdout end ---- ---- doctest::tests::make_test_issues_21299 stdout ---- thread 'doctest::tests::make_test_issues_21299' (56167) panicked at src/librustdoc/doctest/tests.rs:283:5: assertion `left == right` failed left: ("#![allow(unused)]\npub fn main() {\n// fn main\nassert_eq!(2+2, 4);\n}", 2) right: ("#![allow(unused)]\nfn main() {\n// fn main\nassert_eq!(2+2, 4);\n}", 2) stack backtrace: 0: __rustc::rust_begin_unwind 1: core::panicking::panic_fmt 2: core::panicking::assert_failed_inner 3: core::panicking::assert_failed::<(alloc::string::String, usize), (alloc::string::String, usize)> 4: rustdoc::doctest::tests::make_test_issues_21299 5: <rustdoc::doctest::tests::make_test_issues_21299::{closure#0} as core::ops::function::FnOnce<()>>::call_once note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace. ---- doctest::tests::make_test_issues_21299 stdout end ---- ---- doctest::tests::make_test_manual_extern_crate_with_macro_use stdout ---- thread 'doctest::tests::make_test_manual_extern_crate_with_macro_use' (56171) panicked at src/librustdoc/doctest/tests.rs:158:5: assertion `left == right` failed left: ("#![allow(unused)]\n#[macro_use] extern crate asdf;\npub fn main() {\nuse asdf::qwop;\nassert_eq!(2+2, 4);\n}", 2) right: ("#![allow(unused)]\n#[macro_use] extern crate asdf;\nfn main() {\nuse asdf::qwop;\nassert_eq!(2+2, 4);\n}", 2) stack backtrace: 0: __rustc::rust_begin_unwind 1: core::panicking::panic_fmt 2: core::panicking::assert_failed_inner 3: core::panicking::assert_failed::<(alloc::string::String, usize), (alloc::string::String, usize)> 4: rustdoc::doctest::tests::make_test_manual_extern_crate_with_macro_use 5: <rustdoc::doctest::tests::make_test_manual_extern_crate_with_macro_use::{closure#0} as core::ops::function::FnOnce<()>>::call_once note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace. ---- doctest::tests::make_test_manual_extern_crate_with_macro_use stdout end ---- ---- doctest::tests::make_test_named_wrapper stdout ---- thread 'doctest::tests::make_test_named_wrapper' (56172) panicked at src/librustdoc/doctest/tests.rs:355:5: assertion `left == right` failed left: ("#![allow(unused)]\npub fn main() { #[allow(non_snake_case)] fn _doctest_main__some_unique_name() {\nassert_eq!(2+2, 4);\n} _doctest_main__some_unique_name() }", 2) right: ("#![allow(unused)]\nfn main() { #[allow(non_snake_case)] fn _doctest_main__some_unique_name() {\nassert_eq!(2+2, 4);\n} _doctest_main__some_unique_name() }", 2) stack backtrace: 0: __rustc::rust_begin_unwind 1: core::panicking::panic_fmt 2: core::panicking::assert_failed_inner 3: core::panicking::assert_failed::<(alloc::string::String, usize), (alloc::string::String, usize)> 4: rustdoc::doctest::tests::make_test_named_wrapper 5: <rustdoc::doctest::tests::make_test_named_wrapper::{closure#0} as core::ops::function::FnOnce<()>>::call_once note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace. ---- doctest::tests::make_test_named_wrapper stdout end ---- ---- doctest::tests::make_test_opts_attrs stdout ---- thread 'doctest::tests::make_test_opts_attrs' (56174) panicked at src/librustdoc/doctest/tests.rs:178:5: assertion `left == right` failed left: ("#![feature(sick_rad)]\n#[allow(unused_extern_crates)]\nextern crate r#asdf;\npub fn main() {\nuse asdf::qwop;\nassert_eq!(2+2, 4);\n}", 3) right: ("#![feature(sick_rad)]\n#[allow(unused_extern_crates)]\nextern crate r#asdf;\nfn main() {\nuse asdf::qwop;\nassert_eq!(2+2, 4);\n}", 3) stack backtrace: 0: __rustc::rust_begin_unwind 1: core::panicking::panic_fmt 2: core::panicking::assert_failed_inner 3: core::panicking::assert_failed::<(alloc::string::String, usize), (alloc::string::String, usize)> 4: rustdoc::doctest::tests::make_test_opts_attrs 5: <rustdoc::doctest::tests::make_test_opts_attrs::{closure#0} as core::ops::function::FnOnce<()>>::call_once note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace. ---- doctest::tests::make_test_opts_attrs stdout end ---- ---- doctest::tests::make_test_returns_result stdout ---- thread 'doctest::tests::make_test_returns_result' (56175) panicked at src/librustdoc/doctest/tests.rs:341:5: assertion `left == right` failed left: ("#![allow(unused)]\npub fn main() { fn _inner() -> core::result::Result<(), impl core::fmt::Debug> {\nuse std::io;\nlet mut input = String::new();\nio::stdin().read_line(&mut input)?;\nOk::<(), io:Error>(())\n} _inner().unwrap() }", 2) right: ("#![allow(unused)]\nfn main() { fn _inner() -> core::result::Result<(), impl core::fmt::Debug> {\nuse std::io;\nlet mut input = String::new();\nio::stdin().read_line(&mut input)?;\nOk::<(), io:Error>(())\n} _inner().unwrap() }", 2) stack backtrace: 0: __rustc::rust_begin_unwind 1: core::panicking::panic_fmt 2: core::panicking::assert_failed_inner 3: core::panicking::assert_failed::<(alloc::string::String, usize), (alloc::string::String, usize)> 4: rustdoc::doctest::tests::make_test_returns_result 5: <rustdoc::doctest::tests::make_test_returns_result::{closure#0} as core::ops::function::FnOnce<()>>::call_once note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace. ---- doctest::tests::make_test_returns_result stdout end ---- ---- doctest::tests::make_test_no_crate_inject stdout ---- thread 'doctest::tests::make_test_no_crate_inject' (56173) panicked at src/librustdoc/doctest/tests.rs:104:5: assertion `left == right` failed left: ("#![allow(unused)]\npub fn main() {\nuse asdf::qwop;\nassert_eq!(2+2, 4);\n}", 2) right: ("#![allow(unused)]\nfn main() {\nuse asdf::qwop;\nassert_eq!(2+2, 4);\n}", 2) stack backtrace: 0: __rustc::rust_begin_unwind 1: core::panicking::panic_fmt 2: core::panicking::assert_failed_inner 3: core::panicking::assert_failed::<(alloc::string::String, usize), (alloc::string::String, usize)> 4: rustdoc::doctest::tests::make_test_no_crate_inject 5: <rustdoc::doctest::tests::make_test_no_crate_inject::{closure#0} as core::ops::function::FnOnce<()>>::call_once note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace. ---- doctest::tests::make_test_no_crate_inject stdout end ---- 

[rust-log-analyzer]

The job test-various failed! Check out the build log: (web) (plain enhanced) (plain)

Click to see the possible cause of the failure (guessed by this bot)

[rust-bors]

💔 Test for 8d12b29 failed: CI. Failed jobs:

• try - test-various (web logs, extended logs)