-
Notifications
You must be signed in to change notification settings - Fork 4.4k
[8.6.0] Compensate for Windows filesystems lacking junction support #28367
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: release-8.6.0
Are you sure you want to change the base?
Conversation
**Problem**: Bazel fails completely on Windows when using filesystems that don't support junction/reparse point operations (e.g., virtiofs, VirtualBox shared folders, network drives, RAM disks). The fatal error occurs when `ReadSymlinkOrJunction` fails during path resolution (e.g., when Starlark code calls `.realpath`): "Cannot read link: DeviceIoControl: Incorrect function". This causes build analysis to abort completely. Additionally, `CreateJunction` failures when creating convenience symlinks produce cryptic error messages, though these were already non-fatal warnings. Both fail because `DeviceIoControl` returns `ERROR_INVALID_FUNCTION` when the filesystem doesn't implement `FSCTL_GET_REPARSE_POINT` or `FSCTL_SET_REPARSE_POINT` operations. **Proposed solution:** Handle `ERROR_INVALID_FUNCTION` gracefully by treating it as a "not supported" condition rather than a fatal error: 1. in `ReadSymlinkOrJunction` (`file.cc`:592): return `kNotALink` instead of `kError` when `ERROR_INVALID_FUNCTION` occurs. This allows path resolution to continue for non-symlink paths on unsupported filesystems. 2. in `CreateJunction` (`file.cc`:461): return new `kNotSupported` result code when `ERROR_INVALID_FUNCTION` occurs. This produces clear "filesystem does not support junctions" warnings instead of cryptic "Incorrect function" messages. This improves UX but doesn't change behavior (these failures were already non-fatal). This follows the try-first, fallback-on-error pattern (EAFP) used by other major projects when handling unsupported filesystem operations. **Prior art:** - Rust (rust-lang/rust#138133): checks `ERROR_INVALID_FUNCTION`, `ERROR_NOT_SUPPORTED`, and `ERROR_INVALID_PARAMETER` for filesystem operation fallbacks in `std::fs::rename`. - Microsoft STL (microsoft/STL#2077): handles junctions and reparse point errors including `ERROR_INVALID_PARAMETER` with robust fallback logic in `filesystem.cpp`. - Go (golang/go#20506): uses fallback strategies when symlink APIs are unavailable on different Windows versions. - WinFsp (winfsp/winfsp#88): documents that `ERROR_INVALID_FUNCTION` indicates `STATUS_NOT_IMPLEMENTED` for unsupported operations. - Microsoft Learn: recommends checking `FILE_SUPPORTS_REPARSE_POINTS` flag via `GetVolumeInformation`, but try-catch approach is simpler and handles edge cases where detection succeeds but operations fail. **Impact**: - enables Bazel to work on virtiofs, VirtualBox shared folders, RAM disks, and other filesystems that don't support Windows junction operations. - convenience symlinks (bazel-bin, bazel-out, etc.) still won't be created, but now with clearer error messages. **Limitations**: Full junction support would require filesystem-level changes (e.g., virtiofs driver improvements). **Testing:** Tested on Windows 11 VM with host directory mounted via virtiofs, with [rules_pkg](https://github.com/bazelbuild/rules_pkg/blob/6cdaba69ee76463b2b8e97e8d243dbb6115c3aee/toolchains/git/git_configure.bzl#L40). Before change: build analysis aborted with "Cannot read link" fatal error. After change: builds complete successfully with clearer warnings about unsupported junctions for convenience symlinks. Closes bazelbuild#27598. PiperOrigin-RevId: 833360316 Change-Id: I3751602b2bd793c1cee75b7b66fa73c955a72517
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request addresses a fatal error in Bazel on Windows when using filesystems that lack junction support, such as virtiofs. The changes gracefully handle ERROR_INVALID_FUNCTION from DeviceIoControl calls. In ReadSymlinkOrJunction, this error is now treated as if the path is not a link, preventing build analysis from aborting. In CreateJunction, it provides a clearer warning message. The changes are well-contained, logical, and consistent across the native C++ and Java layers. The approach is sound and aligns with practices in other major projects. I find no issues with the implementation.
…28367) **Problem**: Bazel fails completely on Windows when using filesystems that don't support junction/reparse point operations (e.g., virtiofs, VirtualBox shared folders, network drives, RAM disks). The fatal error occurs when `ReadSymlinkOrJunction` fails during path resolution (e.g., when Starlark code calls `.realpath`): "Cannot read link: DeviceIoControl: Incorrect function". This causes build analysis to abort completely. Additionally, `CreateJunction` failures when creating convenience symlinks produce cryptic error messages, though these were already non-fatal warnings. Both fail because `DeviceIoControl` returns `ERROR_INVALID_FUNCTION` when the filesystem doesn't implement `FSCTL_GET_REPARSE_POINT` or `FSCTL_SET_REPARSE_POINT` operations. **Proposed solution:** Handle `ERROR_INVALID_FUNCTION` gracefully by treating it as a "not supported" condition rather than a fatal error: 1. in `ReadSymlinkOrJunction` (`file.cc`:592): return `kNotALink` instead of `kError` when `ERROR_INVALID_FUNCTION` occurs. This allows path resolution to continue for non-symlink paths on unsupported filesystems. 2. in `CreateJunction` (`file.cc`:461): return new `kNotSupported` result code when `ERROR_INVALID_FUNCTION` occurs. This produces clear "filesystem does not support junctions" warnings instead of cryptic "Incorrect function" messages. This improves UX but doesn't change behavior (these failures were already non-fatal). This follows the try-first, fallback-on-error pattern (EAFP) used by other major projects when handling unsupported filesystem operations. **Prior art:** - Rust (rust-lang/rust#138133): checks `ERROR_INVALID_FUNCTION`, `ERROR_NOT_SUPPORTED`, and `ERROR_INVALID_PARAMETER` for filesystem operation fallbacks in `std::fs::rename`. - Microsoft STL (microsoft/STL#2077): handles junctions and reparse point errors including `ERROR_INVALID_PARAMETER` with robust fallback logic in `filesystem.cpp`. - Go (golang/go#20506): uses fallback strategies when symlink APIs are unavailable on different Windows versions. - WinFsp (winfsp/winfsp#88): documents that `ERROR_INVALID_FUNCTION` indicates `STATUS_NOT_IMPLEMENTED` for unsupported operations. - Microsoft Learn: recommends checking `FILE_SUPPORTS_REPARSE_POINTS` flag via `GetVolumeInformation`, but try-catch approach is simpler and handles edge cases where detection succeeds but operations fail. **Impact**: - enables Bazel to work on virtiofs, VirtualBox shared folders, RAM disks, and other filesystems that don't support Windows junction operations. - convenience symlinks (bazel-bin, bazel-out, etc.) still won't be created, but now with clearer error messages. **Limitations**: Full junction support would require filesystem-level changes (e.g., virtiofs driver improvements). **Testing:** Tested on Windows 11 VM with host directory mounted via virtiofs, with [rules_pkg](https://github.com/bazelbuild/rules_pkg/blob/6cdaba69ee76463b2b8e97e8d243dbb6115c3aee/toolchains/git/git_configure.bzl#L40). Before change: build analysis aborted with "Cannot read link" fatal error. After change: builds complete successfully with clearer warnings about unsupported junctions for convenience symlinks. Closes #27598. PiperOrigin-RevId: 833360316 Change-Id: I3751602b2bd793c1cee75b7b66fa73c955a72517 Commit dab96fc Co-authored-by: Rgis Desgroppes <rdesgroppes@gmail.com>
Problem:
Bazel fails completely on Windows when using filesystems that don't support junction/reparse point operations (e.g., virtiofs, VirtualBox shared folders, network drives, RAM disks).
The fatal error occurs when
ReadSymlinkOrJunctionfails during path resolution (e.g., when Starlark code calls.realpath): "Cannot read link: DeviceIoControl: Incorrect function".This causes build analysis to abort completely.
Additionally,
CreateJunctionfailures when creating convenience symlinks produce cryptic error messages, though these were already non-fatal warnings.Both fail because
DeviceIoControlreturnsERROR_INVALID_FUNCTIONwhen the filesystem doesn't implementFSCTL_GET_REPARSE_POINTorFSCTL_SET_REPARSE_POINToperations.Proposed solution:
Handle
ERROR_INVALID_FUNCTIONgracefully by treating it as a "not supported" condition rather than a fatal error:ReadSymlinkOrJunction(file.cc:592): returnkNotALinkinstead ofkErrorwhenERROR_INVALID_FUNCTIONoccurs. This allows path resolution to continue for non-symlink paths on unsupported filesystems.CreateJunction(file.cc:461): return newkNotSupportedresult code whenERROR_INVALID_FUNCTIONoccurs. This produces clear "filesystem does not support junctions" warnings instead of cryptic "Incorrect function" messages. This improves UX but doesn't change behavior (these failures were already non-fatal).This follows the try-first, fallback-on-error pattern (EAFP) used by other major projects when handling unsupported filesystem operations.
Prior art:
ERROR_INVALID_FUNCTION,ERROR_NOT_SUPPORTED, andERROR_INVALID_PARAMETERfor filesystem operation fallbacks instd::fs::rename.<filesystem>:read_symlinkdoes not work with junctions microsoft/STL#2077): handles junctions and reparse point errors includingERROR_INVALID_PARAMETERwith robust fallback logic infilesystem.cpp.ERROR_INVALID_FUNCTIONindicatesSTATUS_NOT_IMPLEMENTEDfor unsupported operations.FILE_SUPPORTS_REPARSE_POINTSflag viaGetVolumeInformation, but try-catch approach is simpler and handles edge cases where detection succeeds but operations fail.Impact:
Limitations:
Full junction support would require filesystem-level changes (e.g., virtiofs driver improvements).
Testing:
Tested on Windows 11 VM with host directory mounted via virtiofs, with rules_pkg. Before change: build analysis aborted with "Cannot read link" fatal error. After change: builds complete successfully with clearer warnings about unsupported junctions for convenience symlinks.
Closes #27598.
PiperOrigin-RevId: 833360316
Change-Id: I3751602b2bd793c1cee75b7b66fa73c955a72517
Commit dab96fc