Make shallow cloning more durable #167
Conversation
|
@timrchavez Please sign the Contributor License Agreement! Click here to manually synchronize the status of this Pull Request. See the FAQ for frequently asked questions. |
|
Assuming I have the test correct, I do not believe the suggested change for #158 will work. The error I'm getting is: |
|
On a shallow clone, I am able to duplicate the error you're seeing. It appears, though, that setting |
|
I use a good ol |
timrchavez
changed the title
If a git-resource is configured to use shallow cloning and new commits are added to the repository that are not considered new versions of the resource (e.g. via a path filter excluding it), a `get` will shallow clone HEAD, which will not be equal to the version ref, and start to break when it then tries to checkout the version ref since there is no longer sufficient history available to find version ref. One solution, which is proposed here, is to always fetch enough depth to checkout the version ref corresponding to that version of the resource.
|
|
||
| git fetch origin refs/notes/*:refs/notes/* | ||
| git checkout -q $ref | ||
| until `git checkout -q $ref`; do |
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
| git fetch origin refs/notes/*:refs/notes/* | ||
| git checkout -q $ref | ||
| until `git checkout -q $ref`; do | ||
| deepen=$((deepen+$depth)) |
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
|
@timrchavez Thank you for signing the Contributor License Agreement! |
|
@timrchavez Do you have time to look at this? No rush, just pinging. :) |
|
@vito Yep. I should have time. I'm not sure about @bhcleek's approach honestly because it requires a configuration change to the GitHub installation itself and based on the documentation it's very resource intensive. I'm open to other suggestions, but I'm not creative enough to think of anything better than a loop which fetches deeper and deeper until it can find its ref. We should do something here, even if it comes with a warning. This could significantly improve build times for us or anyone with very larger (mono)repos (our primary one is something like 1.2GB) by allowing them to safely shallow clone with path filters. |
|
@timrchavez I'm fine with it fetching all the way back to the first commit in the |
If the given ref is not found after a shallow clone, it starts chasing it by deepening the clone in 2-power steps, eventually falling back to a deep clone if a threshold is reached. This should fix concourse#158 / concourse#167.
If the given ref is not found after a shallow clone, it starts chasing it by deepening the clone in steps of 2-power, eventually falling back to a deep clone if a threshold is reached. This should fix concourse#158 / concourse#167.
|
merged #205 |
4 participants
If a git-resource is configured to use shallow cloning and new commits
are added to the repository that are not considered new versions of the
resource (e.g. via a path filter excludes it), a
getwill shallowclone HEAD which will not be equal to the version ref. This will result
in breakage.