Strange case of badFilemode

[EliahKagan]

In recent versions of Git, running git fsck on the rust/rust-lang repository reports two badFilemode info messages, but gix fsck does not report anything.

I don't know if this is a known limitation, absent feature, or bug in gix-fsck, or if it is instead a false positive (and thus a bug) in Git. Examining the trees with git ls-tree or git cat-file -p does not seem to reveal an incorrect mode, but maybe I am missing something, or maybe those are insufficient to show modes other than those that are supposed to appear in a tree object? I am not sure how I should investigate this.

The repository has submodules but whether they are cloned or not doesn't have an affect on this behavior. This is also not a result of how the repository is cloned in terms of git clone vs. gix clone or in terms of HTTPS vs. SSH. The following is after cloning an HTTPS URL with gix clone but the results do not depend on this.

ek@walpurti:~/src/rust (master =)$ git fsck
Checking object directories: 100% (256/256), done.
warning in tree 1cc97948adf5217a9d3097c59a8f7a92b35611b0: badFilemode: contains bad file modes
warning in tree 365664cae1eddce0271440219f4e2f11ed49c3a0: badFilemode: contains bad file modes
Checking objects: 100% (2678358/2678358), done.
Checking connectivity: 2678358, done.
ek@walpurti:~/src/rust (master =)$ git cat-file -p 1cc97948adf5217a9d3097c59a8f7a92b35611b0
100644 blob a7de7ce85593c140267bd3bafa3812859d8f259f    .gitattributes
100644 blob 1428ee6c9bc234acc1cb52d25f54afdcdd5f145c    .gitignore
100644 blob 31db0772cfb6b3f789fde5073b823f3c74908998    .gitmodules
100644 blob a2e3c581eabba32aa116add9c703e3994c988673    .mailmap
100644 blob e3708bc485399fd42b32c6a1c24491771afa1a04    CODE_OF_CONDUCT.md
100644 blob 8fbbd7c4a2e49b0ac2e86dae076ce6e6248ad21e    CONTRIBUTING.md
100644 blob dc9abf84b8e5a4d3b6ab5472883f0997fa0454cc    COPYRIGHT
100644 blob f3fad1296bfd79c76f70d0e113e456f0cdb7051d    Cargo.lock
100644 blob a242f090fbc07e889723c334dcb8689b3314be7b    Cargo.toml
100644 blob 16fe87b06e802f094b3fbb0894b137bca2b16ef1    LICENSE-APACHE
100644 blob 31aa79387f27e730e33d871925e152e35e428031    LICENSE-MIT
100644 blob 61d3c9e1157203f0c4ed5165608d92294eaca808    README.md
100644 blob 5afc6f9bdc0cb9d9d24ce1698c34adcaa5e372b0    RELEASES.md
100644 blob 39be51758068f2031160e7ce3708eb386bcb31ab    config.toml.example
100755 blob eeb8d081d34549f5ca2b19f703bbb4e547264e46    configure
100644 blob 73f8cc1ff68c68bbbbfe6216f4b0f00626701672    rustfmt.toml
040000 tree 757806199afc231c7a7a020f1a4e5ac71707dd55    src
100644 blob f0e3a99037b022118df470c8b7fe27578824e79e    triagebot.toml
100755 blob 7973730ef177cd600aaf4baebe1b40a81ed1b085    x.py
ek@walpurti:~/src/rust (master =)$ git cat-file -p 365664cae1eddce0271440219f4e2f11ed49c3a0
100644 blob 3ab00a6e147fc5b5342ab97b87aa730fff6a9cb8    Cargo.toml
100644 blob c501378bff5491b0bba6cb39a3738f7e7f5845f4    README.md
040000 tree 700c3e7fa316ba0a1c8cd45023a93c19c241e89b    bin
100644 blob 6737086f6e5b0fab8016be51f10defbafbf4ae9a    bootstrap.py
100644 blob 0e941e13676684d8c7b4f8dcfa94500486cba740    bootstrap_test.py
100644 blob a2dca66697d6d641b971ae43700ebdf70e03ee1b    builder.rs
040000 tree e474aaaf48bedd9170f6eb39de82f1262ada29a0    builder
100644 blob 0c16fae01bca788ac7f2f7ba407c8db159da87d2    cache.rs
100644 blob a236edf971fcca2702e0e111688ad217f4d371a8    cc_detect.rs
100644 blob 38810237ef93c206a299c92f0cf4b23c7917ecd0    channel.rs
100644 blob d4016f16fa9d3fe10b908fae38da240d47f40303    check.rs
100644 blob b4e58c06fdea902e39a0ad2f5d838632af388be3    clean.rs
100644 blob 2d60024a22ac4ec6cedf3518fd97c200605c40ee    compile.rs
100644 blob 4fdfb744f381cf8c5707e390ae17372364c156f1    config.rs
100755 blob 7cfc5385e2104987ec773c0e1876f2b1eced09e1    configure.py
100644 blob e64d4c8637d5527c1b0c4142b7b789fd9042d7e9    dist.rs
100644 blob 8cd7fc2c172571b13035d725a7c54383b1523061    doc.rs
100644 blob ffc24367db6e9cf214479ee779ddbd3fb097cb02    flags.rs
100644 blob 65b654fb51929ad8bd429821c3d10efe5c38273d    format.rs
100644 blob f8734ebdf4254213b743dfa125fbc701250addba    install.rs
100644 blob 57153e2ad39f4d7d23bbb8740be0a8be02368acc    job.rs
100644 blob 1fee3fd9ac1d26038726a25287061fd75e7ade5e    lib.rs
100644 blob 8a26adc7ed50109105f8846727cc6640513b1de4    metadata.rs
040000 tree db24a61c4f4c9dc63037071636a6590b67a92180    mk
100644 blob afee154fe714314e614fb30f540219d47b44de37    native.rs
100644 blob 8ff7056e628f32349a86b36e1bda086efd5762be    sanity.rs
100644 blob f2a7d1d00cf2856574486f710d3272a62dbddd6f    test.rs
100644 blob 9fd20386e367bd815c7cb872f1ca9aec4272913b    tool.rs
100644 blob b068c8200acecb86302a162f6b402c0871d70919    toolstate.rs
100644 blob 5fd25981851d4e99af7b99024620843c842cf23a    util.rs
ek@walpurti:~/src/rust (master =)$ gix fsck
ek@walpurti:~/src/rust (master =)$ git version
git version 2.43.0
ek@walpurti:~/src/rust (master =)$ apt list git
Listing... Done
git/noble-updates,noble-security,now 1:2.43.0-1ubuntu7.1 amd64 [installed,automatic]
N: There is 1 additional version. Please use the '-a' switch to see it
ek@walpurti:~/src/rust (master =)$ cargo install --list | grep gitoxide
gitoxide v0.37.0:

The above is in Ubuntu 24.04 LTS but I also verified this on Windows 10. This does not seem to vary by operating system, except that some operating systems that provide git don't have a recent enough version of git for git fsck to report the badFileModes (as I have found to be the case on Ubuntu 22.04 LTS). I believe git is able to detect this since 2.37.3, due to git/git@53602a9, but I have not bisected to verify this. (Downstream git builds usually have backported fixes, but usually only for security bugs.)

In gix-fsck, the check_tree function does not seem to have specific logic to ensure the modes are correct, but it does match on the modes:

gitoxide/gix-fsck/src/lib.rs

Lines 79 to 93 in e249c34

	for entry_ref in tree.entries.iter() {
	match entry_ref.mode.kind() {
	EntryKind::Tree => {
	let tree_id = entry_ref.oid.to_owned();
	tree_ids.push_back(tree_id);
	}
	EntryKind::Blob | EntryKind::BlobExecutable | EntryKind::Link => {
	let blob_id = entry_ref.oid.to_owned();
	if self.seen.insert(blob_id) {
	check_blob(&self.db, &blob_id, &mut self.missing_cb);
	}
	}
	EntryKind::Commit => {
	// Skip submodules as they wouldn't be in this repository!
	}

I had expected the modes to be checked on retrieval, but that doesn't seem to happen. For EntryKind:

gitoxide/gix-object/src/tree/mod.rs

Lines 25 to 26 in e249c34

	/// Note that even though it can represent every valid [mode](EntryMode), it might
	/// loose information due to that as well.

And for EntryMode::kind, which is what produces the mode information matched on in check_tree:

gitoxide/gix-object/src/tree/mod.rs

Line 82 in e249c34

/// Discretize the raw mode into an enum with well-known state while dropping unnecessary details.

So that information is lost, and that any unidentified mode is treated as a submodule (EntryKind::Commit), seems intentional:

gitoxide/gix-object/src/tree/mod.rs

Lines 84 to 98 in e249c34

	let etype = self.0 & IFMT;
	if etype == 0o100000 {
	if self.0 & 0o000100 == 0o000100 {
	EntryKind::BlobExecutable
	} else {
	EntryKind::Blob
	}
	} else if etype == EntryKind::Link as u16 {
	EntryKind::Link
	} else if etype == EntryKind::Tree as u16 {
	EntryKind::Tree
	} else {
	EntryKind::Commit
	}
	}

But it feels like gix-fsck ought to check modes. I'm not sure what the intended behavior is. I'm also not sure how to figure out the cause of the badFilemode diagnostic from recent Git git fsck output.

In a way this might be two questions in one. If there is nothing to be done in gitoxide because file mode validation is outside the scope of the current gix-fsck intended behavior, then this becomes a pure Git question (or a question about the rust-lang/rust repository), which I could ask elsewhere if preferred.

[Byron]

Thanks for bringing this up!

First of all, I checked what git fsck is doing and it looks like it finds a tree that prints its entries correctly, but probably has some extra bits set in the part that isn't printing (see source). Thus it looks OK, but probably isn't. This could be validated by printing with an altered version of Git that doesn't truncate the mode.

gix-fsck is just a demo at this point, as it misses way too much and indeed, should have a proper mode-check instead of relying on the discretized enum. There might even be hidden bugs if code keeps insisting that all modes are correct, or falls back to thinking it's a submodule. Maybe the latter is the least destructive, but maybe there are ways to abuse this as well.

In conclusion, I think this is a bug in Git - it should not discards mode-bits when printing for the purpose truly showing a tree - right now it doesn't look like that exists. gix tree entries 1cc97948adf5217a9d3097c59a8f7a92b35611b0 -e also discretizes modes, but adding another mode that doesn't do that should be easy enough if you'd think that could be a side-effect of your work - using gix as debugging tool is definitely something I envision, and it should be easy to add such features (by design and aspiration).

The gix-fsck could be changed to looks at raw modes to be able to detect such a case, but it's truly optional as it's nothing more than a proof of concept right now. But in theory, it should be easy to add such a check, which really is part of what gix is supposed to do - it should empower people to contribute meaningful changes quite easily.