Fix mishandling of SSH port numbers in llb.Git by aaronlehmann · Pull Request #4069 · moby/buildkit

aaronlehmann · 2023-07-27T01:28:10Z

In creating a host argument to pass into sshutil.SSHKeyScan, the current code loses the port number and will hang if packets to port 22 are blocked. A port number separated by a : should be preserved.

tonistiigi

Do I get it correctly that when the custom port is set then the only valid format is ssh://git@github.com:port/user/repo.git ?

Could we get a test that covers this case to avoid regressions. Unit test just for parsing should be fine.

tonistiigi · 2023-07-31T09:28:24Z

+		// don't do this transformation if the text after the : looks
+		// like a port number.
+		if len(parts) == 2 &&
+			(len(parts[1]) == 0 || parts[1][0] < '0' || parts[1][0] > '9') {


Would it be safer to do strconv.ParseInt in here?

Why would it be safer? I don't think it's quite the same thing, for example it would parse +22 as an int.

iiuc this currently checks the range for first letter after :. Why can't a regular repo name start with a number?

It could, but how would we know it's supposed to be a repo name and not a port number?

I don't know how to know it precisely, but I thought it would at least be safer to check all the letters than just the first one. I didn't consider that ParseInt also accepts the sign.

aaronlehmann · 2023-07-31T15:33:26Z

Do I get it correctly that when the custom port is set then the only valid format is ssh://git@github.com:port/user/repo.git ?

I'm not sure I understand your question precisely, but I don't think that's correct. We parse the protocol out before this with gitutil.ParseProtocol, so the protocol prefix is still optional. This is just addressing the case where : is used a separator between the host and the path. That is now disallowed if there is a port number provided.

I could try to handle URIs like git@github.com:22:moby/buildkit.git - not sure if that's valid, though.

tonistiigi · 2023-07-31T15:48:23Z

I could try to handle URIs like git@github.com:22:moby/buildkit.git - not sure if that's valid, though.

That's what I was wondering as well.

aaronlehmann · 2023-07-31T16:44:04Z

Added a test. Also made this work for URIs like ssh://github.com:22:moby/buildkit.git, though I'm not sure that's a valid format.

jedevc · 2023-07-31T17:16:02Z

Also made this work for URIs like ssh://github.com:22:moby/buildkit.git, though I'm not sure that's a valid format.

$ git clone git@github.com:22:moby/buildkit.git
Cloning into 'buildkit'...
fatal: remote error: 
  is not a valid repository name
Visit https://support.github.com/ for help

I think we should defer to the git cli compatibility here if we can.

We might want to split out the protocol in gitutil.ParseProtocol to return different values for a ssh:// URL vs a scp-like URL, git seems to have different paths for both of these.

aaronlehmann · 2023-07-31T18:29:40Z

I think we should defer to the git cli compatibility here if we can.

Great, this makes things simpler. I've updated the PR to drop support for both a port number and a colon-delimited path at the same time.

tonistiigi · 2023-08-01T07:01:34Z

+		// check for a possible port number after the host
+		if !isNumeric(hostParts[1]) {
+			sshHost = hostParts[0]
+			pathPrefix := strings.Join(hostParts[1:], "/")


This seems to be no-op as hostParts[1:] always only has one item because of the SplitN

This was left over from when it handled multiple :s. Removed.

In creating a host argument to pass into sshutil.SSHKeyScan, the current code loses the port number and will hang if packets to port 22 are blocked. A port number separated by a : should be preserved. Signed-off-by: Aaron Lehmann <alehmann@netflix.com>

aaronlehmann · 2023-08-04T20:32:54Z

@tonistiigi: Does this look good now?

jedevc · 2023-08-11T16:09:53Z

+		{
+			remote:           "ssh://github.com:22:moby/buildkit.git",
+			expectedProtocol: gitutil.SSHProtocol,
+			expectedSSHHost:  "github.com",


Expected host here should be github.com:22:moby if we want to follow the same behavior as git:

$ git clone ssh://git@github.com:moby:22/buildkit.git Cloning into 'buildkit'... ssh: Could not resolve hostname github.com:moby:22: Name or service not known fatal: Could not read from remote repository. Please make sure you have the correct access rights and the repository exists.

jedevc · 2023-08-11T16:13:22Z

@aaronlehmann @tonistiigi what do you think of the approach in master...jedevc:buildkit:wip-gitutil? I had a bit of a play around.

Essentially a lot of this issue seems to be that ParseProtocol and callers confuse between the git@ scp-style URLs and the ssh://-style. Port numbers are only valid in ssh:// style, so the logic to do the exact host and path splitting should be pushed down into ParseProtocol where we actually determine if it's an scp-style URL.

(I think this solves the original issue?)

aaronlehmann · 2023-08-11T17:24:28Z

@jedevc: I really like that approach. Using url.Parse when possible makes a lot of sense. Happy to close this PR in favor of an alternative solution.

aaronlehmann · 2023-08-14T15:27:14Z

Closing in favor of #4142

aaronlehmann requested a review from tonistiigi July 28, 2023 18:42

tonistiigi reviewed Jul 31, 2023

View reviewed changes

aaronlehmann force-pushed the ssh-port-numbers branch 2 times, most recently from 3d99e08 to d7afafc Compare July 31, 2023 16:47

aaronlehmann force-pushed the ssh-port-numbers branch from d7afafc to 6624adb Compare July 31, 2023 18:29

tonistiigi reviewed Aug 1, 2023

View reviewed changes

aaronlehmann force-pushed the ssh-port-numbers branch from 6624adb to eeb6f87 Compare August 1, 2023 15:02

jedevc self-requested a review August 11, 2023 14:27

jedevc reviewed Aug 11, 2023

View reviewed changes

jedevc mentioned this pull request Aug 13, 2023

Refactor git ssh url parsing #4142

Merged

aaronlehmann closed this Aug 14, 2023

jedevc mentioned this pull request Oct 11, 2023

git urls are incorrectly transformed from relative to absolute (unable to clone git repo from server when the repo is relative to the git user's home dir) #4322

Closed

jedevc mentioned this pull request May 21, 2024

docker do not honor custom ssh port in ADD instruction while cloning git repositories docker/buildx#2468

Closed

Conversation

aaronlehmann commented Jul 27, 2023

Uh oh!

tonistiigi left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

aaronlehmann commented Jul 31, 2023

Uh oh!

tonistiigi commented Jul 31, 2023

Uh oh!

aaronlehmann commented Jul 31, 2023

Uh oh!

jedevc commented Jul 31, 2023

Uh oh!

aaronlehmann commented Jul 31, 2023

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

aaronlehmann commented Aug 4, 2023

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jedevc commented Aug 11, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

aaronlehmann commented Aug 11, 2023

Uh oh!

aaronlehmann commented Aug 14, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

jedevc commented Aug 11, 2023 •

edited

Loading