Add a new download-ci-llvm = if-unchanged option and enable it by default for profile = codegen

[LuuuXXX]

Three tasks have been implemented here.

Add a new download-ci-llvm = if-unchange option and enable if by
default for profile = codegen.

Include all build artifacts by traversing the llvm-project build output,
Keep the downloadable llvm the same state as if you have just run a full
source build.

After selecting the codegen profile during ./x.py setup, the submodule
will be automatically downloaded.

It should be noted that：
if you are using the 'codegen' profile, be aware that the default value changed. otherwise, you can ignore this.

Resolves #110087

[rustbot]

r? @Mark-Simulacrum

(rustbot has picked a reviewer for you, use r? to override)

[rustbot]

This PR changes config.example.toml. If appropriate, please also update CONFIG_CHANGE_HISTORY in src/bootstrap/lib.rs and change-id in config.example.toml.

This PR changes src/bootstrap/defaults/config.codegen.toml. If appropriate, please also update config.compiler.toml so the defaults are in sync.

[rustbot]

This PR changes src/bootstrap/defaults/config.codegen.toml. If appropriate, please also update config.compiler.toml so the defaults are in sync.

This PR changes config.example.toml. If appropriate, please also update CONFIG_CHANGE_HISTORY in src/bootstrap/lib.rs and change-id in config.example.toml.

[LuuuXXX]

By the way, triagebot.toml should be updated because the bootstrap codebase is reorganized. The tips above seem out of date.

[onur-ozkan]

Current PR ID(which is 116881) should be appended to this constant:

rust/src/bootstrap/src/lib.rs

Lines 73 to 81 in 020d008

	/// Keeps track of major changes made to the bootstrap configuration.
	///
	/// These values also represent the IDs of the PRs that caused major changes.
	/// You can visit `https://github.com/rust-lang/rust/pull/{any-id-from-the-list}` to
	/// check for more details regarding each change.
	///
	/// If you make any major changes (such as adding new values or changing default values), please
	/// ensure that the associated PR ID is added to the end of this list.
	pub const CONFIG_CHANGE_HISTORY: &[usize] = &[115898];

This should be replaced with the current PR ID:

rust/config.example.toml

Lines 22 to 33 in 020d008

	# Keeps track of major changes made to this configuration.
	#
	# This value also represents ID of the PR that caused major changes. Meaning,
	# you can visit github.com/rust-lang/rust/pull/{change-id} to check for more details.
	#
	# A 'major change' includes any of the following
	# - A new option
	# - A change in the default values
	#
	# If `change-id` does not match the version that is currently running,
	# `x.py` will prompt you to update it and check the related PR for more details.
	change-id = 115898

[onur-ozkan]

I still need some work to confirm the second task, maybe a separate PR is needed.

I think we can handle it in this PR

[rustbot]

This PR modifies src/bootstrap/src/core/config. If appropriate, please also update CONFIG_CHANGE_HISTORY in src/bootstrap/src/lib.rs and change-id in config.example.toml.

This PR modifies src/bootstrap/defaults. If appropriate, please also update CONFIG_CHANGE_HISTORY in src/bootstrap/src/lib.rs and change-id in config.example.toml.

[bors]

☔ The latest upstream changes (presumably #116998) made this pull request unmergeable. Please resolve the merge conflicts.

[onur-ozkan]

Can you please rebase and push the commit with a more descriptive message?

Commit history still looks inappropriate.

[LuuuXXX]

reworded it

[onur-ozkan]

Can you squash your commits into "Fix issue #110087" one ? Following commits alone doesn't make too much sense as far as I see. Specially the 2nd one.

[onur-ozkan]

Thank you for the fixes.

CC #110087
I still need some work to confirm the second task, maybe a separate PR is needed.

PR description seems to be outdated, it would be nice if you update it :) I will queue up the PR anyway.

[onur-ozkan]

@bors r+ rollup=iffy

[bors]

📌 Commit 1b8dee1 has been approved by onur-ozkan

It is now in the queue for this repository.

[bors]

⌛ Testing commit 1b8dee1 with merge 750c2ec...

[bors]

☀️ Test successful - checks-actions
Approved by: onur-ozkan
Pushing 750c2ec to master...

[rust-timer]

Finished benchmarking commit (750c2ec): comparison URL.

Overall result: no relevant changes - no action needed

@rustbot label: -perf-regression

Instruction count

This benchmark run did not return any relevant results for this metric.

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	1.0%	[0.5%, 1.5%]	3
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.7%	[-0.8%, -0.6%]	4
Improvements ✅ (secondary)	-3.7%	[-3.7%, -3.7%]	1
All ❌✅ (primary)	0.0%	[-0.8%, 1.5%]	7

Cycles

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	0.4%	[0.4%, 0.4%]	1
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.4%	[-0.4%, -0.4%]	1
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	0.0%	[-0.4%, 0.4%]	2

Binary size

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-0.0%	[-0.0%, -0.0%]	4
All ❌✅ (primary)	-	-	0

Bootstrap: 663.473s -> 664.216s (0.11%)
Artifact size: 308.82 MiB -> 308.79 MiB (-0.01%)

[RalfJung]

I'm getting a WARNING pointing me to this PR. Do I need to do something? Or can I keep my existing config? It's unclear to me. The PR description says there's a new config, but why does everyone have to be alerted of its existence?

[RalfJung]

Also this list is unordered now, that seems wrong?

[LuuuXXX]

update change-id = 116881 instead may be help.
AFAIK This is ordered by merge time to keep the source code synchronized with the config file.

[RalfJung]

update change-id = 116881 instead may be help.

That will silence the warning, sure. But the reason the warning is shown must be that I should do something before silencing it. There's no point in having people mindlessly increment a number every week just to get rid of the latest warning.

[onur-ozkan]

Also this list is unordered now, that seems wrong?

rust/src/bootstrap/src/lib.rs

Lines 78 to 80 in 1db4b12

	/// If you make any major changes (such as adding new values or changing default values),
	/// please ensure that the associated PR ID is added to the end of this list.
	/// This is necessary because the list must be sorted by the merge date.

I'm getting a WARNING pointing me to this PR. Do I need to do something? Or can I keep my existing config? It's unclear to me

rust/src/bootstrap/defaults/config.codegen.toml

Lines 12 to 13 in 1db4b12

	# build llvm from source
	download-ci-llvm = "if-unchanged"

PR updated the default value here. I am pretty sure we have people who wants to know this change.

[RalfJung]

Then the PR description needs to say: "if you are coming here from x.py, here's what do do: if you are using the 'codegen' profile, be aware that the default value changed. otherwise, you can ignore this".

Currently I'm coming here clueless and wasting 5min trying to figure out if I should do something, before giving up, frustrated.

[LuuuXXX]

That's my bad, I will add the description for this PR later

[RalfJung]

For this PR I am also confused why we have if-available and if-unchanged as two different options. When would I ever want if-available? Shouldn't if-unchanged just be the default everywhere, since it will "do the right thing" in most situations?

[LuuuXXX]

When would I ever want if-available?

If there is no need to modify llvm locally, if-available may be able to give you faster build times. Since you can use Rust CI built LLVM instead of locally building it. and if-unchanged will consider whether there are local modifications in llvm.

Shouldn't if-unchanged just be the default everywhere, since it will "do the right thing" in most situations?

Is it similar to this? https://github.com/orgs/rust-lang/projects/35/views/1?pane=issue&itemId=40227883

[RalfJung]

If there is no need to modify llvm locally, if-available may be able to give you faster build times.

Have you measured it? How long does the change-check take?

We also have true for skipping all the checks, after all.

[LuuuXXX]

Not yet, the change-check is based on git diff-index it will not take too much time.