Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use git-filter-repo to strip historical large IPFS binaries #367

Open
livid opened this issue Jun 11, 2024 · 2 comments
Open

Use git-filter-repo to strip historical large IPFS binaries #367

livid opened this issue Jun 11, 2024 · 2 comments

Comments

@livid
Copy link
Member

livid commented Jun 11, 2024

Result of git filter-repo --analyze from 73a7e66:

=== All paths by reverse accumulated size ===
Format: unpacked size, packed size, date deleted, path name
   239504981  108685405 2022-10-11 Planet/IPFS/go-ipfs-executables/ipfs-amd64
   238619056  104103282 2022-10-11 Planet/IPFS/go-ipfs-executables/ipfs-arm64
   110305152   39602143 2022-10-11 Planet/Assets.xcassets/IPFS-GO.dataset/ipfs
   110437216   38533437 2022-10-11 Planet/Assets.xcassets/IPFS-GO-ARM.dataset/ipfs
    67460213   33836608 <present>  Planet/IPFS/ipfs-executables/ipfs-amd64-0.28.bin
    67460213   33836608 <present>  Planet/IPFS/go-ipfs-executables/ipfs-amd64-0.28.bin
    65355701   31854693 <present>  Planet/IPFS/ipfs-executables/ipfs-arm64-0.28.bin
    65355701   31854693 <present>  Planet/IPFS/go-ipfs-executables/ipfs-arm64-0.28.bin
    62964533   28783701 2022-10-11 Planet/IPFS/go-ipfs-executables/ipfs-amd64.bin
    62437893   27342472 2022-10-11 Planet/IPFS/go-ipfs-executables/ipfs-arm64.bin
    11005024   11003931 <present>  Planet/Labs/Icon Gallery/Icons/tier2.zip
     9225100    9225233 <present>  Planet/Labs/Icon Gallery/Icons/tier3.zip
    11586800    6243872 2022-07-20 Planet/IPFS/go-ipfs-executables/fs-repo-migrations-amd64
    11190496    5992917 2022-07-20 Planet/IPFS/go-ipfs-executables/fs-repo-migrations-arm64
     3648387    3599562 <present>  Screenshots/planetable.eth.png
     3028464    1863744 <present>  Planet/IPFS/fs-repo-migrations/fs-repo-14-to-15_amd64
     3000352    1765206 <present>  Planet/IPFS/fs-repo-migrations/fs-repo-14-to-15_arm64
     2720336    1666619 <present>  Planet/IPFS/fs-repo-migrations/fs-repo-12-to-13_amd64
     2698528    1653893 <present>  Planet/IPFS/fs-repo-migrations/fs-repo-13-to-14_amd64
     2670480    1563164 <present>  Planet/IPFS/fs-repo-migrations/fs-repo-12-to-13_arm64
     2652384    1552732 <present>  Planet/IPFS/fs-repo-migrations/fs-repo-13-to-14_arm64
     1438642    1352107 <present>  Screenshots/vitalik.eth.png
     1208368    1163191 <present>  Technotes/Images/xcode15_build_issue.png
    31522127    1135696 <present>  Planet.xcodeproj/project.pbxproj
    ...

du -h: size of a fresh clone:

...
117M	./.git/lfs
518M	./.git
657M	.

Run this command to strip the largest IPFS files that are not in LFS:

git filter-repo \
--path Planet/IPFS/go-ipfs-executables/ipfs-amd64 \
--path Planet/IPFS/go-ipfs-executables/ipfs-arm64 \
--path Planet/Assets.xcassets/IPFS-GO.dataset/ipfs \
--path Planet/Assets.xcassets/IPFS-GO-ARM.dataset/ipfs \
--path Planet/IPFS/ipfs-executables/ipfs-amd64-0.28.bin \
--path Planet/IPFS/go-ipfs-executables/ipfs-amd64-0.28.bin \
--path Planet/IPFS/ipfs-executables/ipfs-arm64-0.28.bin \
--path Planet/IPFS/go-ipfs-executables/ipfs-arm64-0.28.bin \
--path Planet/IPFS/go-ipfs-executables/ipfs-amd64.bin \
--path Planet/IPFS/go-ipfs-executables/ipfs-arm64.bin \
--invert-paths

du -h: after running that command:

...
117M	./.git/lfs
182M	./.git
321M	.
@livid
Copy link
Member Author

livid commented Jun 11, 2024

This issue does not block #365

@livid
Copy link
Member Author

livid commented Jun 11, 2024

After #365 is merged, tested, and released as an Insider build, I will return to this issue to clean this tech debt.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant