Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Checkout of contributed-docs/build not sparse #203

Closed
kbarros opened this issue Dec 21, 2023 · 4 comments
Closed

Checkout of contributed-docs/build not sparse #203

kbarros opened this issue Dec 21, 2023 · 4 comments

Comments

@kbarros
Copy link
Member

kbarros commented Dec 21, 2023

It appears that our docs build is not actually performing a sparse checkout of https://github.com/SunnySuite/SunnyContributed/tree/main/contributed-docs/build. The output below, from running docs/make.jl, indicates 21MB are downloaded, even though the files are small. I wonder if the history has some large files that were removed. cc @ddahlbom.

Initialized empty Git repository in /Users/kbarros/.julia/dev/Sunny/docs/contributed-tmp/.git/
Updating origin
remote: Enumerating objects: 489, done.
remote: Counting objects: 100% (81/81), done.
remote: Compressing objects: 100% (66/66), done.
remote: Total 489 (delta 26), reused 43 (delta 11), pack-reused 408
Receiving objects: 100% (489/489), 21.16 MiB | 1.78 MiB/s, done.
Resolving deltas: 100% (229/229), done.
From https://github.com/SunnySuite/SunnyContributed
 * [new branch]      main       -> origin/main
From https://github.com/SunnySuite/SunnyContributed
 * branch            main       -> FETCH_HEAD
@kbarros
Copy link
Member Author

kbarros commented Dec 21, 2023

Can we store the contributed docs build as a tar-gzip and then download from https://raw.githubusercontent.com/SunnySuite/SunnyContributed/main/contributed-docs/build.tgz

For the download: https://docs.julialang.org/en/v1/stdlib/Downloads/#Downloads.download
For extracting: https://github.com/JuliaIO/GZip.jl and https://github.com/JuliaIO/Tar.jl.

Alternatively Zlib supports a multi-file archive out of the box, but seems to be less maintained? https://github.com/fhs/ZipFile.jl/blob/master/src/Zlib.jl

@kbarros
Copy link
Member Author

kbarros commented Dec 21, 2023

Another option is to store the docs build on a separate branch, say docs-build. Then it can be cloned efficiently using:

git clone --single-branch --branch docs-build git@github.com:SunnySuite/SunnyContributed.git

If it ever gets too big, we can wipe out the history on docs-build and recreate.

@ddahlbom
Copy link
Member

I think I favor your first suggestion -- just make a tar and download it. No need to periodically wipe histories to keep the system behaving. I'll try it out.

@kbarros
Copy link
Member Author

kbarros commented Jan 3, 2024

Fixed in #213.

@kbarros kbarros closed this as completed Jan 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants