New format_args!() and fmt::Arguments implementation

[m-ou-se]

Part of #99012

This is a new implementation of format_args!() and fmt::Arguments. With this implementation, fmt::Arguments is only two pointers in size. (Instead of six, before.) This makes it the same size as a &str and makes it fit in a register pair.

---

This fmt::Arguments can store a &'static str without any indirection or additional storage. This means that simple cases like print_fmt(format_args!("hello")) are now just as efficient for the caller as print_str("hello"), as shown by this example:

code:

fn main() { println!("Hello, world!"); }

before:

main:  sub rsp, 56  lea rax, [rip + .Lanon_hello_world]  mov qword ptr [rsp + 8], rax  mov qword ptr [rsp + 16], 1  mov qword ptr [rsp + 24], 8  xorps xmm0, xmm0  movups xmmword ptr [rsp + 32], xmm0  lea rdi, [rsp + 8]  call qword ptr [rip + std::io::stdio::_print]  add rsp, 56  ret

after:

main:  lea rsi, [rip + .Lanon_hello_world]  mov edi, 29  jmp qword ptr [rip + std::io::stdio::_print]

(panic!("Hello, world!"); shows a similar change.)

---

This implementation stores all static information as just a single (byte) string, without any indirection:

code:

format_args!("Hello, {name:-^20}!")

lowering before:

fmt::Arguments::new_v1_formatted( &["Hello, ", "!\n"], &args, &[ Placeholder { position: 0usize, flags: 3355443245u32, precision: format_count::Implied, width: format_count::Is(20u16), }, ], )

lowering after:

fmt::Arguments::new( b"\x07Hello, \xc3-\x00\x00\xc8\x14\x00\x02!\n\x00", &args, )

This saves a ton of pointers and simplifies the expansion significantly, but does mean that individual pieces (e.g. "Hello, " and "!\n") cannot be reused. (Those pieces are often smaller than a pointer to them, though, in which case reusing them is useless.)

---

The details of the new representation are documented in library/core/src/fmt/mod.rs.

---

Diagram of the data structure after this change:

Original data structure

[m-ou-se]

@bors try @rust-timer queue

[rust-bors]

☀️ Try build successful (CI)
Build commit: 6e6ba94 (6e6ba949d24fbfbd9cd48ca4c98adf59fbd04482, parent: a7b3715826827677ca8769eb88dc8052f43e734b)

[rust-timer]

Finished benchmarking commit (6e6ba94): comparison URL.

Overall result: ❌✅ regressions and improvements - please read the text below

Benchmarking this pull request means it may be perf-sensitive – we'll automatically label it not fit for rolling up. You can override this, but we strongly advise not to, due to possible changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please do so in sufficient writing along with @rustbot label: +perf-regression-triaged. If not, please fix the regressions and do another perf run. If its results are neutral or positive, the label will be automatically removed.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

	mean	range	count
Regressions ❌ (primary)	0.7%	[0.1%, 5.8%]	26
Regressions ❌ (secondary)	0.6%	[0.1%, 1.3%]	44
Improvements ✅ (primary)	-0.7%	[-4.3%, -0.1%]	109
Improvements ✅ (secondary)	-1.7%	[-38.2%, -0.0%]	93
All ❌✅ (primary)	-0.5%	[-4.3%, 5.8%]	135

Max RSS (memory usage)

Results (primary -1.5%, secondary -0.6%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	2.2%	[2.2%, 2.2%]	1
Regressions ❌ (secondary)	3.7%	[1.0%, 6.7%]	12
Improvements ✅ (primary)	-1.6%	[-6.0%, -0.5%]	31
Improvements ✅ (secondary)	-2.6%	[-7.9%, -0.7%]	25
All ❌✅ (primary)	-1.5%	[-6.0%, 2.2%]	32

Cycles

Results (primary -0.5%, secondary -4.3%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	4.8%	[3.4%, 6.2%]	2
Regressions ❌ (secondary)	8.8%	[2.6%, 18.8%]	6
Improvements ✅ (primary)	-3.1%	[-5.0%, -2.1%]	4
Improvements ✅ (secondary)	-10.3%	[-39.4%, -2.1%]	13
All ❌✅ (primary)	-0.5%	[-5.0%, 6.2%]	6

Binary size

Results (primary -0.7%, secondary -1.3%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	0.5%	[0.0%, 1.4%]	4
Regressions ❌ (secondary)	3.2%	[0.0%, 7.5%]	12
Improvements ✅ (primary)	-0.8%	[-3.3%, -0.0%]	129
Improvements ✅ (secondary)	-1.7%	[-23.6%, -0.0%]	123
All ❌✅ (primary)	-0.7%	[-3.3%, 1.4%]	133

Bootstrap: 476.631s -> 471.922s (-0.99%)
Artifact size: 391.32 MiB -> 388.56 MiB (-0.70%)

[m-ou-se]

Ooh that's pretty good :D

[m-ou-se]

Pretty much everything looks like a great improvement. Not only number of instructions executed, but also memory usage and binary size. 🎉

Only two significant negative results:

1. "image-0.25.6 opt incr-patched:println" with almost +6% instructions:u.

Looking at the detailed results, it looks like that's all LLVM. Probably because llvm got more optimization opportunities. That's not necessarily a bad thing.

2. The fmt-write-str runtime benchmark with over +12% instructions:u.

This could be concerning, but I can't seem to fully replicate it locally.

If I recompile and run this benchmark 100 times in both nightly and with this PR, I do get this interesting result though:

With the nightly compiler, the results vary, with many measurements clustered close to 25ms but also many around 40ms. With this PR, the results are very consistent, all clustered around 27ms. (Update: It's around 26ms now, after a minor optimization.)

So, the median result is worse, but the average is better.

My guess is that the indirection (a slice of string slices) can make things unpredictable, as the strings aren't always in the optimal place for caching. The lack of indirection in the new version then makes it much more predictable. This is just a guess though.

[m-ou-se]

@bors try @rust-timer queue

[WaffleLapkin]

all my nits are pretty minor, so r=me with or without them

View changes since this review

[jdonszelmann]

also looked it over, r=me,wafflelapkin after nits

[m-ou-se]

@bors r=wafflelapkin,jdonszelmann rollup=never

[bors]

📌 Commit cfbdc2c has been approved by wafflelapkin,jdonszelmann

It is now in the queue for this repository.

[bors]

⌛ Testing commit cfbdc2c with merge 503dce3...

[bors]

☀️ Test successful - checks-actions
Approved by: wafflelapkin,jdonszelmann
Pushing 503dce3 to main...

[github-actions]

What is this?

This is an experimental post-merge analysis report that shows differences in test outcomes between the merged PR and its parent PR.

Comparing 0186755 (parent) -> 503dce3 (this PR)

Test differences

Show 202 test diffs

202 doctest diffs were found. These are ignored, as they are noisy.

Test dashboard

Run

cargo run --manifest-path src/ci/citool/Cargo.toml -- \ test-dashboard 503dce33e2e2a5d2fe978b2723ab2a994cc27472 --output-dir test-dashboard

And then open test-dashboard/index.html in your browser to see an overview of all executed tests.

Job duration changes

1. dist-apple-various: 3283.0s -> 4299.8s (+31.0%)
2. aarch64-apple: 9944.8s -> 7106.3s (-28.5%)
3. x86_64-rust-for-linux: 2505.4s -> 3146.5s (+25.6%)
4. dist-i686-mingw: 11724.5s -> 9515.8s (-18.8%)
5. x86_64-gnu-gcc: 3064.8s -> 3594.9s (+17.3%)
6. dist-x86_64-solaris: 4916.8s -> 5715.6s (+16.2%)
7. aarch64-gnu-debug: 3918.9s -> 4512.8s (+15.2%)
8. x86_64-gnu-tools: 3291.3s -> 3769.9s (+14.5%)
9. pr-check-1: 1489.6s -> 1700.0s (+14.1%)
10. i686-gnu-1: 7428.5s -> 8319.6s (+12.0%)

How to interpret the job duration changes?

Job durations can vary a lot, based on the actual runner instance
that executed the job, system noise, invalidated caches, etc. The table above is provided
mostly for t-infra members, for simpler debugging of potential CI slow-downs.

[rust-timer]

Finished benchmarking commit (503dce3): comparison URL.

Overall result: ❌✅ regressions and improvements - please read the text below

Our benchmarks found a performance regression caused by this PR.
This might be an actual regression, but it can also be just noise.

Next Steps:

• If the regression was expected or you think it can be justified,
  please write a comment with sufficient written justification, and add
  @rustbot label: +perf-regression-triaged to it, to mark the regression as triaged.
• If you think that you know of a way to resolve the regression, try to create
  a new PR with a fix for the regression.
• If you do not understand the regression or you think that it is just noise,
  you can ask the @rust-lang/wg-compiler-performance working group for help (members of this group
  were already notified of this PR).

@rustbot label: +perf-regression
cc @rust-lang/wg-compiler-performance

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

	mean	range	count
Regressions ❌ (primary)	0.7%	[0.1%, 5.7%]	17
Regressions ❌ (secondary)	0.6%	[0.1%, 1.1%]	40
Improvements ✅ (primary)	-0.7%	[-4.4%, -0.1%]	120
Improvements ✅ (secondary)	-1.6%	[-38.5%, -0.0%]	106
All ❌✅ (primary)	-0.5%	[-4.4%, 5.7%]	137

Max RSS (memory usage)

Results (primary -1.5%, secondary -1.6%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	3.8%	[1.4%, 7.0%]	3
Regressions ❌ (secondary)	3.8%	[1.2%, 5.8%]	8
Improvements ✅ (primary)	-2.2%	[-6.0%, -0.6%]	23
Improvements ✅ (secondary)	-3.1%	[-7.3%, -0.6%]	29
All ❌✅ (primary)	-1.5%	[-6.0%, 7.0%]	26

Cycles

Results (primary -2.3%, secondary 0.4%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	5.2%	[3.4%, 6.9%]	2
Regressions ❌ (secondary)	7.6%	[1.6%, 18.5%]	19
Improvements ✅ (primary)	-4.4%	[-7.9%, -1.5%]	7
Improvements ✅ (secondary)	-11.0%	[-40.0%, -1.8%]	12
All ❌✅ (primary)	-2.3%	[-7.9%, 6.9%]	9

Binary size

Results (primary -0.8%, secondary -1.3%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	0.5%	[0.0%, 1.4%]	4
Regressions ❌ (secondary)	3.0%	[0.0%, 6.9%]	12
Improvements ✅ (primary)	-0.8%	[-3.3%, -0.0%]	129
Improvements ✅ (secondary)	-1.7%	[-23.6%, -0.0%]	123
All ❌✅ (primary)	-0.8%	[-3.3%, 1.4%]	133

Bootstrap: 476.356s -> 473.309s (-0.64%)
Artifact size: 391.04 MiB -> 388.34 MiB (-0.69%)