cli: renku session start fails for git remote

[lokijuhy]

Describe the bug

I tried to use renku session start on my GeeMap evaluation project, and it errored with "Ahhhh you have found a bug".

❯ git clone git@renkulab.io:laura.kinkead1/geemap-evaluation.git
Cloning into 'geemap-evaluation'...
remote: Enumerating objects: 91, done.
remote: Counting objects: 100% (31/31), done.
remote: Compressing objects: 100% (23/23), done.
remote: Total 91 (delta 13), reused 20 (delta 8), pack-reused 60
Receiving objects: 100% (91/91), 24.39 KiB | 2.71 MiB/s, done.
Resolving deltas: 100% (24/24), done.
❯ cd geemap-evaluation
❯ renku session start
Ahhhhhhhh! You have found a bug. 🐞

1. Open an issue by typing "open";
2. Print human-readable information by typing "print";
3. See the full traceback without submitting details (default: "ignore").

Please select an action by typing its name (open, print, ignore) [ignore]: open

Details

I can successfully run renku session start on the Renku-stories/geemap project I have cloned in the same directory.

Ralf suspects this is caused by the . in the project's namespace.

Renku version: 1.3.1
OS: Darwin (Darwin Kernel Version 21.4.0: Fri Mar 18 00:46:32 PDT 2022; root:xnu-8020.101.4~15/RELEASE_ARM64_T6000)
Python: 3.9.12
Docker Desktop: 4.8.2

Traceback

Traceback (most recent call last):
  File "[...]/site-packages/docker/api/client.py", line 268, in _raise_for_status
    response.raise_for_status()
  File "[...]/site-packages/requests/models.py", line 960, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.41/images/laura.kinkead1/geemap-evaluation:56c4aa5/json

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "[...]/site-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "[...]/site-packages/docker/api/image.py", line 251, in inspect_image
    return self._result(
  File "[...]/site-packages/docker/api/client.py", line 274, in _result
    self._raise_for_status(response)
  File "[...]/site-packages/docker/api/client.py", line 270, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "[...]/site-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
docker.errors.ImageNotFound: 404 Client Error for http+docker://localhost/v1.41/images/laura.kinkead1/geemap-evaluation:56c4aa5/json: Not Found ("no such image: laura.kinkead1/geemap-evaluation:56c4aa5: No such image: laura.kinkead1/geemap-evaluation:56c4aa5")

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "[...]/site-packages/docker/api/client.py", line 268, in _raise_for_status
    response.raise_for_status()
  File "[...]/site-packages/requests/models.py", line 960, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: http+docker://localhost/v1.41/images/create?tag=56c4aa5&fromImage=laura.kinkead1%2Fgeemap-evaluation

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "[...]/site-packages/renku/core/session/docker.py", line 79, in find_image
    _ = self.docker_client().images.pull(image_name)
  File "[...]/site-packages/docker/models/images.py", line 444, in pull
    pull_log = self.client.api.pull(
  File "[...]/site-packages/docker/api/image.py", line 428, in pull
    self._raise_for_status(response)
  File "[...]/site-packages/docker/api/client.py", line 270, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "[...]/site-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
docker.errors.APIError: 500 Server Error for http+docker://localhost/v1.41/images/create?tag=56c4aa5&fromImage=laura.kinkead1%2Fgeemap-evaluation: Internal Server Error ("Get "https://laura.kinkead1/v2/": Failed to lookup host: laura.kinkead1")

[Panaetius]

this is likely due to https://github.com/SwissDataScienceCenter/renku-python/blob/develop/renku/core/session/utils.py#L53 not working for git+ssh urls where the user name contains a dot (like git@renkulab.io:laura.kinkead1/geemap-evaluation.git), in which case the endpoint url resolves to laura.kinkead1 instead of renkulab.io

[Panaetius]

I can't reproduce this on Linux, maybe someone with a mac could take a look

[lokijuhy]

I heard from a user who had an issue running renku session start, and I think it may be the same issue. The traceback looks the same. He's operating on WSL (and with no dot in the user name). He sent me the bug report:

Describe the bug

Ran renku session start after upgrading from 1.3.0. Worked when running from 1.3.0, but same command gave an empty URL.

Details

Time of OS is completely wrong. Time is Mon June 20 17:10:00 UTC 2022.
This was run on WSL2-Debian.

Renku version: 1.4.0
OS: Linux (#1 SMP Fri Apr 2 22:23:49 UTC 2021)
Python: 3.9.2

Traceback

Traceback (most recent call last):
  File "[...]/docker/api/client.py", line 268, in _raise_for_status
    response.raise_for_status()
  File "[...]/requests/models.py", line 960, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.41/images/registry.renkulab.io/osaeedi/msc-thesis_paleoclimate_ml:06243a0/json
 
During handling of the above exception, another exception occurred:
 
Traceback (most recent call last):
  File "[...]/renku/core/session/docker.py", line 75, in find_image
    _ = self.docker_client().images.get(image_name)
  File "[...]/docker/models/images.py", line 314, in get
    return self.prepare_model(self.client.api.inspect_image(name))
  File "[...]/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "[...]/docker/api/image.py", line 251, in inspect_image
    return self._result(
  File "[...]/docker/api/client.py", line 274, in _result
    self._raise_for_status(response)
  File "[...]/docker/api/client.py", line 270, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "[...]/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
docker.errors.ImageNotFound: 404 Client Error for http+docker://localhost/v1.41/images/registry.renkulab.io/osaeedi/msc-thesis_paleoclimate_ml:06243a0/json: Not Found ("no such image: [registry.renkulab.io/osaeedi/msc-thesis_paleoclimate_ml:06243a0:](http://registry.renkulab.io/osaeedi/msc-thesis_paleoclimate_ml:06243a0:) No such image: [registry.renkulab.io/osaeedi/msc-thesis_paleoclimate_ml:06243a0](http://registry.renkulab.io/osaeedi/msc-thesis_paleoclimate_ml:06243a0)")
 
During handling of the above exception, another exception occurred:
 
Traceback (most recent call last):
  File "[...]/docker/api/client.py", line 268, in _raise_for_status
    response.raise_for_status()
  File "[...]/requests/models.py", line 960, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: http+docker://localhost/v1.41/images/create?tag=06243a0&fromImage=registry.renkulab.io%2Fosaeedi%2Fmsc-thesis_paleoclimate_ml
 
During handling of the above exception, another exception occurred:
 
Traceback (most recent call last):
  File "[...]/renku/ui/cli/exception_handler.py", line 129, in main
    result = super().main(*args, **kwargs)
  File "[...]/renku/ui/cli/exception_handler.py", line 93, in main
    return super().main(*args, **kwargs)
  File "[...]/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "[...]/click/core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "[...]/click/core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "[...]/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "[...]/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "[...]/renku/ui/cli/session.py", line 179, in start
    session_start_command()
  File "[...]/renku/command/command_builder/command.py", line 296, in execute
    output = context["click_context"].invoke(self._operation, *args, **kwargs)
  File "[...]/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "[...]/inject/__init__.py", line 342, in injection_wrapper
    return sync_func(*args, **kwargs)
  File "[...]/renku/core/session/session.py", line 89, in session_start
    if not provider_api.find_image(image_name, config):
  File "[...]/renku/core/session/docker.py", line 79, in find_image
    _ = self.docker_client().images.pull(image_name)
  File "[...]/docker/models/images.py", line 444, in pull
    pull_log = self.client.api.pull(
  File "[...]/docker/api/image.py", line 428, in pull
    self._raise_for_status(response)
  File "[...]/docker/api/client.py", line 270, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "[...]/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
docker.errors.APIError: 500 Server Error for http+docker://localhost/v1.41/images/create?tag=06243a0&fromImage=registry.renkulab.io%2Fosaeedi%2Fmsc-thesis_paleoclimate_ml: Internal Server Error ("Head https://registry.renkulab.io/v2/osaeedi/msc-thesis_paleoclimate_ml/manifests/06243a0: denied: access forbidden")
`

[Panaetius]

I think they need to do renku login

[lokijuhy]

I tried doing renku login before I ran renku session start (the same example as in the issue description), and I still got the error.

Or are you saying that you think the bug I have is different from the user's I added?

Traceback (most recent call last):
  File "[...]/site-packages/docker/api/client.py", line 268, in _raise_for_status
    response.raise_for_status()
  File "[...]/site-packages/requests/models.py", line 960, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.41/images/laura.kinkead1/geemap-evaluation:56c4aa5/json

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "[...]/site-packages/renku/core/session/docker.py", line 75, in find_image
    _ = self.docker_client().images.get(image_name)
  File "[...]/site-packages/docker/models/images.py", line 314, in get
    return self.prepare_model(self.client.api.inspect_image(name))
  File "[...]/site-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "[...]/site-packages/docker/api/image.py", line 251, in inspect_image
    return self._result(
  File "[...]/site-packages/docker/api/client.py", line 274, in _result
    self._raise_for_status(response)
  File "[...]/site-packages/docker/api/client.py", line 270, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "[...]/site-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
docker.errors.ImageNotFound: 404 Client Error for http+docker://localhost/v1.41/images/laura.kinkead1/geemap-evaluation:56c4aa5/json: Not Found ("no such image: laura.kinkead1/geemap-evaluation:56c4aa5: No such image: laura.kinkead1/geemap-evaluation:56c4aa5")

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "[...]/site-packages/docker/api/client.py", line 268, in _raise_for_status
    response.raise_for_status()
  File "[...]/site-packages/requests/models.py", line 960, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: http+docker://localhost/v1.41/images/create?tag=56c4aa5&fromImage=laura.kinkead1%2Fgeemap-evaluation

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "[...]/site-packages/renku/ui/cli/exception_handler.py", line 128, in main
    result = super().main(*args, **kwargs)
  File "[...]/site-packages/renku/ui/cli/exception_handler.py", line 92, in main
    return super().main(*args, **kwargs)
  File "[...]/site-packages/click/core.py", line 1062, in main
    rv = self.invoke(ctx)
  File "[...]/site-packages/click/core.py", line 1668, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "[...]/site-packages/click/core.py", line 1668, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "[...]/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "[...]/site-packages/click/core.py", line 763, in invoke
    return __callback(*args, **kwargs)
  File "[...]/site-packages/renku/ui/cli/session.py", line 92, in start
    session_start_command()
  File "[...]/site-packages/renku/command/command_builder/command.py", line 291, in execute
    output = context["click_context"].invoke(self._operation, *args, **kwargs)
  File "[...]/site-packages/click/core.py", line 763, in invoke
    return __callback(*args, **kwargs)
  File "[...]/site-packages/inject/__init__.py", line 342, in injection_wrapper
    return sync_func(*args, **kwargs)
  File "[...]/site-packages/renku/core/session/session.py", line 89, in session_start
    if not provider_api.find_image(image_name, config):
  File "[...]/site-packages/renku/core/session/docker.py", line 79, in find_image
    _ = self.docker_client().images.pull(image_name)
  File "[...]/site-packages/docker/models/images.py", line 444, in pull
    pull_log = self.client.api.pull(
  File "[...]/site-packages/docker/api/image.py", line 428, in pull
    self._raise_for_status(response)
  File "[...]/site-packages/docker/api/client.py", line 270, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "[...]/site-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
docker.errors.APIError: 500 Server Error for http+docker://localhost/v1.41/images/create?tag=56c4aa5&fromImage=laura.kinkead1%2Fgeemap-evaluation: Internal Server Error ("Get "https://laura.kinkead1/v2/": Failed to lookup host: laura.kinkead1")

[Panaetius]

is it a private project?