Lock download file to avoid races #4606
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
2 tasks
konstin force-pushed the konsti/lock-downloads branch from 4b0e0d9 to 19c23a7 Compare 
See #TBD for details. This is one of two possible implementations, the easier one is using a temporary directory or a unique filename to ensure processes don't collide. Using locks has less platform support, but OTOH means that we don't create stray temp files that don't get cleaned up (because we never know if one of those files is actually used by another process). This does not fix #TBD completely. It only fixed the downloaded, parallel `rustup-init` calls still fail at the transaction stage. This PR introduces a `LockedFile` primitive that's modeled after the uv type of the same name. It's currently very simple and locks forever. We can extend it with a timeout that makes rustup exit after a certain duration and print an error message, instead of an inscrutable waiting without any message.
konstin force-pushed the konsti/lock-downloads branch from 19c23a7 to e69fb23 Compare 
weihanglo reviewed Nov 20, 2025
src/download/mod.rs Outdated
Comment on lines 50 to 52
| // Otherwise, create a temporary file with 777 permissions, to handle races between | ||
| // processes running under different UIDs (e.g., in Docker containers). We must set | ||
| // permissions _after_ creating the file, to override the `umask`. |
Member
There was a problem hiding this comment.
Interesting. I guess Cargo's file locks also have the same issue?
Member
Would be great if you can a link to it :) |
Contributor Author
| The uv type: https://github.com/astral-sh/uv/blob/fc0cf90795dbfa961f22fc0934a53afc9c13c349/crates/uv-fs/src/lib.rs#L672. I have a branch that adds async timeouts, but that's in review still. |
src/download/mod.rs Outdated
| }; | ||
| if let Err(err) = file | ||
| .as_file() | ||
| .set_permissions(std::fs::Permissions::from_mode(0o777)) |
There was a problem hiding this comment.
Can the permissions be cached in a static variable?
Contributor Author
There was a problem hiding this comment.
The method is a no-op constructor.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit. This suggestion is invalid because no changes were made to the code. Suggestions cannot be applied while the pull request is closed. Suggestions cannot be applied while viewing a subset of changes. Only one suggestion per line can be applied in a batch. Add this suggestion to a batch that can be applied as a single commit. Applying suggestions on deleted lines is not supported. You must change the existing code in this line in order to create a valid suggestion. Outdated suggestions cannot be applied. This suggestion has been applied or marked resolved. Suggestions cannot be applied from pending reviews. Suggestions cannot be applied on multi-line comments. Suggestions cannot be applied while the pull request is queued to merge. Suggestion cannot be applied right now. Please check back later.
See #4607 for details.
This is one of two possible implementations, the easier one is using a temporary directory or a unique filename to ensure processes don't collide. Using locks has less platform support, but OTOH means that we don't create stray temp files that don't get cleaned up (because we never know if one of those files is actually used by another process).
This does not fix #4607 completely. It only fixed the downloaded, parallel
rustup-initcalls still fail at the transaction stage.This PR introduces a
LockedFileprimitive that's modeled after the uv type of the same name. It's currently very simple and locks forever. We can extend it with a timeout that makes rustup exit after a certain duration and print an error message, instead of an inscrutable waiting without any message.