Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: Namespace issues causing files to appear as empty in Android potentially causing dataloss #794

Open
EDLLT opened this issue Oct 22, 2024 · 3 comments
Labels
bug Something isn't working

Comments

@EDLLT
Copy link

EDLLT commented Oct 22, 2024

Describe the bug

When using Obsidian Git plugin on Android, certain files exhibit a filesystem metadata inconsistency after performing git operations across large commit differences.

Specific symptoms:

  1. Files appear empty when checked with ls -la or file commands (showing 0 bytes)
  2. However, the actual file contents are present and accessible using cat command(using termux/adb)
  3. The affected files appear to be random and vary between operations
  4. This issue seems to occur more frequently with:
    • Larger git history
    • Larger gaps between commit checkouts
    • When pulling many changes after a hard reset to an older commit

Relevant errors

No direct error messages are shown, but filesystem inspection reveals:

$ ls -la LM\ Studio.md 
-rw-rw----. 1 root everybody 0 Oct 26 16:02 'LM Studio.md'
$ file LM\ Studio.md 
LM Studio.md: empty
$ cat LM\ Studio.md 
[Shows actual file contents]

Steps to reproduce

  1. Have a git repository with a large commit history in Obsidian on Android
  2. Using Termux git, perform a hard reset to a much older commit
  3. Using the Obsidian git plugin, pull to get back to the latest commit
  4. Inspect files using terminal commands (via Termux or ADB)
  5. Some files will show the metadata inconsistency described above

Expected Behavior

  • File metadata should correctly reflect the actual file contents
  • File size in ls -la should match the actual content size
  • File should not be reported as empty when it contains content

Additional context

  • This appears to be a filesystem metadata synchronization issue
  • Possibly related to how isomorphic-git handles file operations on Android
  • May be caused by race conditions during bulk file operations
  • Issue frequency increases with the scale of git operations being performed

Operating system

Android
(Issue was reproduced on my Xiaomi Pad 6 and Samsung device)

Installation Method

None

Plugin version

2.27.0

@EDLLT EDLLT added the bug Something isn't working label Oct 22, 2024
@EDLLT EDLLT changed the title [Bug]: Mass renaming normal file names to names with emojis causing data loss [Bug]: Mass renaming normal directory names to names with emojis causes data loss Oct 22, 2024
@EDLLT EDLLT changed the title [Bug]: Mass renaming normal directory names to names with emojis causes data loss [Bug]: Namespace issues causing files to appear as empty in Android potentially causing dataloss Oct 26, 2024
@EDLLT
Copy link
Author

EDLLT commented Oct 26, 2024

~/.../Obsidian_TestVault/AI $ ls -la LM\ Studio.md 
-rw-rw----. 1 root everybody 0 Oct 26 16:02 'LM Studio.md'
~/.../Obsidian_TestVault/AI $ file LM\ Studio.md 
LM Studio.md: empty
~/.../Obsidian_TestVault/AI $ cat LM\ Studio.md 
In an ideal case extreme case, the maximum gpu offload layers is 28
26 GPU offload layers for stable usage(higher than that, the model might crash)

## Memory Leak
When you fail to load an LLM model within LM Studio, it still takes up vram. This in return causes you to not be able to fully load the model within the GPU.
The solution is to close LM Studio and run it successfully the first time whilst checking `nvidia-smi`

## Cuda vs Vulkan
You can load up your models in LM Studio using either Cuda or vulkan. Since we have an nvidia GPU, using Cuda is preferable and faster(we need to make sure we have [[Nvidia#^108bb3|Cuda installed]])

In here, this is the termux output(similar results when using adb)
My guess is that this is a namespace issue as I had faced very similar issues in the past when I had root.

The interesting thing is that this does not occur to all files, but rather, it occurs randomly(and the larger the commit history, the more likely it'd happen)

TL;DR: The android system sees the file as empty(and expectedly, git), but using cat proves otherwise.
This is probably being caused by the way isomorphic git is writing and updating files

@EDLLT
Copy link
Author

EDLLT commented Oct 26, 2024

Okay, after thinking about it for a while, perhaps it's being caused by multiple writes to the same file when checking out from a large commit history size to the current file. Maybe there are multiple writes that are happening to the same file which causes a race condition somehow?

@EDLLT
Copy link
Author

EDLLT commented Oct 26, 2024

@Vinzent03 Hey, I am not sure whether if this is considered to be an Obsidian plugin issue or if it's an Obsidian issue with their Adapter and Vault APIs.
Should I open an issue in Obsidian's forums instead?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant