`exp`: CI runs require unshallow clone (fetch-depth: >1)

[dberenbaum]

Summary: dvc exp commands will not run in CML or other Github actions (and probably the same on other Git servers?) because they will only fetch the head commit by default, which is insufficient for internal git merges. The solution is in treeverse/dvc#7547 (comment).

Not sure where to document this or even whether it belongs in DVC, CML, or both. @casperdcl Maybe you have thoughts?

[casperdcl]

For CML workflows, cml ci --unshallow is best.

In non-CML cases (probably worth mentioning in dvc.org docs), unshallow clones are needed (actions/checkout with fetch-depth: 0)

[dberenbaum]

I never noticed cml ci before, looks great. I'm still unsure where it would be best to recommend using this when running dvc exp commands on cml or other ci workflows.

[daavoo]

What about creating a new section in https://dvc.org/doc/user-guide/troubleshooting , catching the specific exception on DVC side, and make it point to that URL

[jorgeorpinel]

Which exp commands/ scenarios exactly do not run in shallow clones? E.g. I assume exp init works.

I'm guessing mostly exp branch/apply (due to internal git merge)? If so we can make a small note in those refs and in https://dvc.org/doc/user-guide/experiment-management/persisting-experiments

Thanks

[dberenbaum]

No, unfortunately even exp run (see the discussion linked from the original comment) needs a non-shallow clone.

[jorgeorpinel]

OK. Hmmm I'm not sure we need to much of a change other than perhaps a small note in https://dvc.org/doc/user-guide/experiment-management/experiments-overview and https://dvc.org/doc/command-reference/exp (/init). Seems like an edge case, or have you noticed this being a common issue out there outside of CML?

https://cml.dev/doc/cml-with-dvc should probably note this and recommend cml ci? Cc @casperdcl

[daavoo]

Seems like an edge case, or have you noticed this being a common issue out there outside of CML?

It's an issue in many CI scenarios (with or without CML) because the default behavior in both GitHub (https://github.com/actions/checkout) and GitLab (https://docs.gitlab.com/ee/ci/large_repositories/#shallow-cloning) is a shallow clone.