Add support for treeless and/or blobless clones

[jemerald]

What feature do you want to see added?

In multi-branch pipeline setup, add options to support treeless and/or blobless clones

See what they are: https://github.blog/2020-12-21-get-up-to-speed-with-partial-clone-and-shallow-clone/

Why?

We are a relatively large repo where a full clone takes several minutes and transfers more than 2GB of data.

And to avoid invisible merge issue, we have opt for building pull requests with the strategy of Merging the pull request with the current target branch revision.

Unfortunately this means that the shallow clone option is not available to us as merging is not possible (and the plugin defaults back to doing a full clone).

The (relatively) new treeless/blobless git feature should greatly improve clone performance for large repos while still cater for merge strategy. I'd even recommend to use the blobless clone as the default option as it does not compromise on feature (unlike shallow clone).

Here's a comparison of transfer size and resulting folder size for our repo:

Clone method	Network transfer size	Directory size
full	2.57 GiB	2.9G
shallow	88.82 MiB	380M
blobless	91.21 MiB + 88.24 MiB	505M
treeless	20.88 MiB + 768.26 KiB + 88.24 MiB	405M

Upstream changes

No response

[KalleOlaviNiemitalo]

Have you tried a reference repository as described in Jenkins hints for large git repos?

[jemerald]

yes, reference repo does speed up the pipeline checkout on Jenkins master (down to ~1m for the first build of each new branch), but does not help with checkout scm in the pipeline as it runs on ephemeral k8s slaves