Skip to content

[main] scripts/checkout.sh: prevent ls-remote from matching multiple tags#375

Merged
thaJeztah merged 1 commit into
docker:mainfrom
thaJeztah:fix_checkout
Jul 17, 2024
Merged

[main] scripts/checkout.sh: prevent ls-remote from matching multiple tags#375
thaJeztah merged 1 commit into
docker:mainfrom
thaJeztah:fix_checkout

Conversation

@thaJeztah
Copy link
Copy Markdown
Member

git ls-remote's argument 1 is a glob 2, and matches anything ending with the given string. This is problematic if multiple tags or branches end with the given pattern. In containerd's case, this returns both tags for the main module ("refs/tags/v1.7.19") and # the API module ("refs/tags/api/v1.7.19").

To prevent both of those being found, we check if the given reference starts with a "v"; if it does, we can assume it's a tag, and prefix the pattern with "refs/tags/" to make it less ambiguous.

We're using a case statement here to avoid introducing Bashisms.

Before this patch:

make REF=v1.7.19 checkout
# ...
Initialized empty Git repository in /Users/thajeztah/go/src/github.com/docker/containerd-packaging/src/github.com/containerd/containerd/.git/
git -C src/github.com/containerd/containerd remote add origin "https://github.com/containerd/containerd.git"
./scripts/checkout.sh src/github.com/containerd/containerd "v1.7.19"
+ SRC=src/github.com/containerd/containerd
+ REF=v1.7.19
+ REF_FETCH=v1.7.19
++ git -C src/github.com/containerd/containerd ls-remote --refs --heads --tags origin v1.7.19
++ awk '{print $2}'
+ REF='refs/tags/api/v1.7.19
refs/tags/v1.7.19'
+ '[' -n 'refs/tags/api/v1.7.19
refs/tags/v1.7.19' ']'
+ REF_FETCH='refs/tags/api/v1.7.19
refs/tags/v1.7.19:refs/tags/api/v1.7.19
refs/tags/v1.7.19'
+ git -C src/github.com/containerd/containerd fetch --update-head-ok --depth 1 origin 'refs/tags/api/v1.7.19
refs/tags/v1.7.19:refs/tags/api/v1.7.19
refs/tags/v1.7.19'
fatal: invalid refspec 'refs/tags/api/v1.7.19
refs/tags/v1.7.19:refs/tags/api/v1.7.19
refs/tags/v1.7.19'
make: *** [checkout] Error 128

With this patch:

make REF=v1.7.19 checkout
# ...
Initialized empty Git repository in /Users/thajeztah/go/src/github.com/docker/containerd-packaging/src/github.com/containerd/containerd/.git/
git -C src/github.com/containerd/containerd remote add origin "https://github.com/containerd/containerd.git"
./scripts/checkout.sh src/github.com/containerd/containerd "v1.7.19"
+ SRC=src/github.com/containerd/containerd
+ REF=v1.7.19
+ REF_FETCH=v1.7.19
+ REF_GLOB=v1.7.19
+ case $REF_GLOB in
+ REF_GLOB=refs/tags/v1.7.19
++ git -C src/github.com/containerd/containerd ls-remote --refs --heads --tags origin refs/tags/v1.7.19
++ awk '{print $2}'
+ REF=refs/tags/v1.7.19
+ '[' -n refs/tags/v1.7.19 ']'
+ REF_FETCH=refs/tags/v1.7.19:refs/tags/v1.7.19
+ git -C src/github.com/containerd/containerd fetch --update-head-ok --depth 1 origin refs/tags/v1.7.19:refs/tags/v1.7.19
remote: Enumerating objects: 6397, done.
remote: Counting objects: 100% (6397/6397), done.
remote: Compressing objects: 100% (5114/5114), done.
Receiving objects: 100% (6397/6397), 10.09 MiB | 13.94 MiB/s, done.
remote: Total 6397 (delta 1376), reused 3349 (delta 816), pack-reused 0
Resolving deltas: 100% (1376/1376), done.
From https://github.com/containerd/containerd
 * [new tag]         v1.7.19    -> v1.7.19
+ git -C src/github.com/containerd/containerd checkout -q refs/tags/v1.7.19

- A picture of a cute animal (not mandatory but encouraged)

@thaJeztah thaJeztah self-assigned this Jul 16, 2024
Comment thread scripts/checkout.sh Outdated
"v"*)
ref_glob="refs/tags/$ref_glob"
;;
esac
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't be indented?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh! yes it can; not sure what happened. Let me shfmt it.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed 👍

git ls-remote's <pattern> argument [1] is a glob [2], and matches anything
ending with the given string. This is problematic if multiple tags or
branches end with the given pattern. In containerd's case, this returns
both tags for the main module ("refs/tags/v1.7.19") and 	# the API module
("refs/tags/api/v1.7.19").

To prevent both of those being found, we check if the given reference starts
with a "v"; if it does, we can assume it's a tag, and prefix the pattern with
"refs/tags/" to make it less ambiguous.

We're using a case statement here to avoid introducing Bashisms.

[1]: https://git-scm.com/docs/git-ls-remote#Documentation/git-ls-remote.txt-ltpatternsgt82308203
[2]: https://git-scm.com/docs/gitglossary#Documentation/gitglossary.txt-glob

Before this patch:

    make REF=v1.7.19 checkout
    # ...
    Initialized empty Git repository in /Users/thajeztah/go/src/github.com/docker/containerd-packaging/src/github.com/containerd/containerd/.git/
    git -C src/github.com/containerd/containerd remote add origin "https://github.com/containerd/containerd.git"
    ./scripts/checkout.sh src/github.com/containerd/containerd "v1.7.19"
    + SRC=src/github.com/containerd/containerd
    + REF=v1.7.19
    + REF_FETCH=v1.7.19
    ++ git -C src/github.com/containerd/containerd ls-remote --refs --heads --tags origin v1.7.19
    ++ awk '{print $2}'
    + REF='refs/tags/api/v1.7.19
    refs/tags/v1.7.19'
    + '[' -n 'refs/tags/api/v1.7.19
    refs/tags/v1.7.19' ']'
    + REF_FETCH='refs/tags/api/v1.7.19
    refs/tags/v1.7.19:refs/tags/api/v1.7.19
    refs/tags/v1.7.19'
    + git -C src/github.com/containerd/containerd fetch --update-head-ok --depth 1 origin 'refs/tags/api/v1.7.19
    refs/tags/v1.7.19:refs/tags/api/v1.7.19
    refs/tags/v1.7.19'
    fatal: invalid refspec 'refs/tags/api/v1.7.19
    refs/tags/v1.7.19:refs/tags/api/v1.7.19
    refs/tags/v1.7.19'
    make: *** [checkout] Error 128

With this patch:

    make REF=v1.7.19 checkout
    # ...
    Initialized empty Git repository in /Users/thajeztah/go/src/github.com/docker/containerd-packaging/src/github.com/containerd/containerd/.git/
    git -C src/github.com/containerd/containerd remote add origin "https://github.com/containerd/containerd.git"
    ./scripts/checkout.sh src/github.com/containerd/containerd "v1.7.19"
    + SRC=src/github.com/containerd/containerd
    + REF=v1.7.19
    + REF_FETCH=v1.7.19
    + REF_GLOB=v1.7.19
    + case $REF_GLOB in
    + REF_GLOB=refs/tags/v1.7.19
    ++ git -C src/github.com/containerd/containerd ls-remote --refs --heads --tags origin refs/tags/v1.7.19
    ++ awk '{print $2}'
    + REF=refs/tags/v1.7.19
    + '[' -n refs/tags/v1.7.19 ']'
    + REF_FETCH=refs/tags/v1.7.19:refs/tags/v1.7.19
    + git -C src/github.com/containerd/containerd fetch --update-head-ok --depth 1 origin refs/tags/v1.7.19:refs/tags/v1.7.19
    remote: Enumerating objects: 6397, done.
    remote: Counting objects: 100% (6397/6397), done.
    remote: Compressing objects: 100% (5114/5114), done.
    Receiving objects: 100% (6397/6397), 10.09 MiB | 13.94 MiB/s, done.
    remote: Total 6397 (delta 1376), reused 3349 (delta 816), pack-reused 0
    Resolving deltas: 100% (1376/1376), done.
    From https://github.com/containerd/containerd
     * [new tag]         v1.7.19    -> v1.7.19
    + git -C src/github.com/containerd/containerd checkout -q refs/tags/v1.7.19

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
@thaJeztah thaJeztah merged commit 0194852 into docker:main Jul 17, 2024
@thaJeztah thaJeztah deleted the fix_checkout branch July 17, 2024 00:10
@thaJeztah thaJeztah mentioned this pull request Jul 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants