Automatically restart from local devfile when devfile failed to be resolved at startup

[l0rd]

Is your enhancement related to a problem? Please describe

There are situation where che-server is not able to resolve a devfile from a git repository using the git service API. A typical example is when it's not GitHub, GitLab, Bitbucket or Azure DevOps repository or when SSH URL is used.

Describe the solution you'd like

Project clone, after cloning the git repository, updates the DevWorkspace object if both conditions are met:

• The env variable DEVFILE_RESOLUTION=failed is set
• A devfile exist at the root of the main project

More information

This is part of this epic: #21951

[amisevsk]

One difficulty in this is that it appears that the dashboard is still modifying the underlying devfile in converting it to a DevWorkspace, so it's not clear how to patch the DevWorkspace from the project-clone container without pulling in a lot of Che-specific logic.

I believe dashboard could avoid this by injecting e.g. environment variables into the editor contribution DevWorkspaceTemplate instead (environment variables from contributions are merged into the main container).

I've created issues

• Avoid any modifications to the devfile in conversion to a DevWorkspace #22614
• Add ability to "bootstrap" devworkspaces in project-clone devfile/devworkspace-operator#1192

For related tasks for this issue.

[amisevsk]

PR devfile/devworkspace-operator#1193 should address the DWO-side of this issue. In testing these changes with Che, I found a few sticking points, however.

To test, my setup was

• Basic Che install on OpenShift
• Manually-created PAT secret in my user's namespace
• Using a private test repository that contains the Che Dashboard's devfile

I found that #22614 isn't terribly important to getting this to work, as the injected environment variables, etc., are still injected into the end result because they're added to the contribution container as well (so the resulting "flattened" DevWorkspace is generally identical to the successfully-generated one).

To make it work, I had to work around the following issues manually (in addition to adding the new attribute):

• When using a private repository in the dashboard, if a devfile cannot be resolved the workspace starts from the empty devfile with no projects added, so I had to manually add my project to the DevWorkspace after it was created
• The DevWorkspace is named empty-<chars>, matching an empty-devfile workspace (it might make more sense to try extract a repository name)

[l0rd]

@amisevsk can we close this one or is still something to check? Also this was the last subtask of Support reading devfiles from private repos other than GitHub/GitLab/Bitbucket/Azure · Issue #21951 · eclipse/che. Have you had the chance to test with a Git service that is not GitHub, Gitlab, Bitbucket and Azure Devops? cc @dmytro-ndp

[amisevsk]

Regarding non-GitHub/Gitlab/Bitbucket/etc. repositories, the DWO-side changes should work for anything so long as the workspace is started with git credentials suitable for cloning from that repository. As for how to test it directly, I don't have any such repositories available, though.

At this point, we have a foundation we can potentially build off of (the controller.devfile.io/bootstrap-devworkspace: true attribute), but there's more to be done make this work for users automatically:

• The big one is that something (e.g. the dashboard) needs to start using the attribute when appropriate; if we fail to resolve a devfile at startup, we need to start a workspace that uses the attribute and includes the project. This could just be a template based off the empty workspace:

  schemaVersion: 2.2.0
  metadata:
    name: <name>
  components:
    - name: universal-developer-image
      container:
        image: quay.io/devfile/universal-developer-image:ubi8-latest
  # ^^^ standard empty workspace devfile ^^^
  attributes:
    controller.devfile.io/bootstrap-devworkspace: true # Clone repo and start from internal devfile if available
  projects:
    - name: <specified project name>
      git:
        remotes:
          origin: <specified repository>

• We also need to figure out a flow for when the bootstrap process fails -- what does this look like, and is it understandable to the user? E.g. if the credentials are invalid, the project will still not be cloned and the workspace won't be restarted; they'll just have the project-clone errors log.
• There are some minor polish issues to work out (e.g. if we fail to resolve a devfile we name the workspace empty-xxxx instead of something based off the URL; if we have a git server that has to use this functionality it could mean having three empty-xxxx workspaces and no way to remember which one is which)

[l0rd]

In the description of this issue I proposed to trigger try the restart from local devfile when the env variable DEVFILE_RESOLUTION=failed is found. We can use an attribute too if you prefer. But yes, dashboard should add this attribute if the resolution fails (and the DWO/cloner should remove it when it resolves the devfile?). The mechanism is in place DWO/cloner side right? If that's the case we need to create an issue for team B with a clear explanation on what project cloner currently expects.

If the process fails I am expecting the workspace startup to fail and the dashboard showing the error and a message as it does it for normal flow no?

For workspace naming: dashboard currently infers the name from the last string of the GIT+SSH URL (it doesn't name it "empty").

For testing @dmytro-ndp can help you with a gogs repository or you can get one at https://sr.ht/ in minutes