Set Partial Clone flags on sparse-checkout#4044
Conversation
There was a problem hiding this comment.
I found one checkout-path issue that looks like it prevents the new auto-filter behavior from meeting the sparse-checkout intent when users haven't also supplied clone flags such as --sparse.
Want to dig deeper? The full session log is attached to this Buildkite build. Download the session file and open a new pi session with it:
Download the buildsworth logs from build 2623, then answer my questions about the findings.
| if hasPartialCloneFilter(gitCloneFlags) { | ||
| e.shell.Commentf("Sparse checkout is configured and BUILDKITE_GIT_CLONE_FLAGS already contains a --filter (preserving user-supplied filter).") | ||
| } else { | ||
| gitCloneFlags = append(gitCloneFlags, "--filter=blob:none") |
There was a problem hiding this comment.
Blocking: This auto-add also runs when the clone flags don't include --sparse (or another flag that suppresses the initial checkout). In that case git clone --filter=blob:none still checks out the full worktree before we call sparse-checkout set --cone, so Git hydrates blobs outside the requested sparse paths up front and this doesn't deliver the intended partial-clone saving. The new integration coverage includes --sparse, which is the case where the filter works; could we either add --sparse along with the filter when supported, or only add the filter when the clone flags already prevent the full initial checkout?
Description
When
BUILDKITE_GIT_SPARSE_CHECKOUT_PATHSis set, the agent now injects--filter=blob:noneinto git clone so blobs outside the sparse set aren't downloaded up front. The filter is only injected when the user hasn't already supplied their own--filter(git takes the last--filteron the command line, so stacking ours after would silently override theirs) and when the local git actually supportssparse-checkout set --cone(>= 2.27). On older git, the agent falls back to a plain full checkout — no partial clone, no--coneattempt.The sparse-checkout decision is resolved once at the top of
defaultCheckoutPhaseviaplanSparseCheckout, which runs and validates thegit --versionat most once per checkout and only when sparse paths are configured.Changes
sparseCheckoutPlanstruct andplanSparseCheckoutfunction to perform git version check only once before clone startssingle-call sparse-checkout set --conelanded in 2.27; 2.26 only accepted--coneoninit).--filter=blob:noneintogitCloneFlagsonly whensparsePlan.supportedand the user hasn't set their own--filter; pass the plan through tosetupSparseCheckout.Testing
go test ./...). Buildkite employees may check this if the pipeline has run automatically.go tool gofumpt -extra -w .)Disclosures / Credits
Claude code drafted all the tests and helped diagnose build failures. I applied and reviewed all changes.