Allow runtime install of tools

[viceice]

Make all possible tools runtime installable.

TODO

• prepare-tool: install tool base deps as root #121
• Cache dir / volume #197
• Restructuring of folders/links #213
• bundler Support install-tool for Bundler and Cocoapods #251
• cocoapods Support install-tool for Bundler and Cocoapods #251
• composer
• docker feat(docker): convert to v2 tool #651
• dotnet: runtime install #118
• elixir feat(erlang): migrate erlang to v2 tools #376
• erlang: runtime install #318
• add flutter tool #200
• git-lfs feat(git-lfs): convert to v2 #652
• git runtime install #653
• golang: runtime install #119
• gradle
• hashin (pip_requirements) feat(python): simlify install and add hashin, pip-tools, pipenv #639
• helm: runtime install #199
• java
• Runtime installation of Maven #178
• nix feat(nix): convert to v2 #573
• node: runtime install #217
• npm
• pipenv feat(python): simlify install and add hashin, pip-tools, pipenv #639
• pip-tools (pip-compile) feat(python): simlify install and add hashin, pip-tools, pipenv #639
• php: runtime install #120
• pnpm
• poetry
• feat(powershell): convert to v2 tool #633
• feat: convert python to a v2 tool #404
• ruby: runtime install #475
• rust: runtime install #571
• sbt
• scala
• swift: runtime install #576
• fix: terraform non-root install #181, test: install non-root terraform #198
• yarn

[rarkins]

I don't think it really matters much whether it's in /home/user or /usr/local however /usr/local would probably be better for OpenShift, where the UID is random.

[viceice]

We like to implement a new binarySource=install where renovate install tools dynamically at runtime.

Use-case is to have smaller docker image used in kubernets, which can download missing tools on demand.

ref #96

renovatebot/docker-buildpack#144 (comment)

[rarkins]

@viceice should we break these into separate issues to research and complete one by one? I'd like to know early if we have any "blockers", too

[rarkins]

Thinking it through, we'll need:

1. OS pre-requisites which need to be built into the base image, mostly stuff that requires apt
2. "prebuilt" versions of most languages (we do not want to compile them at install time) and some tools. For these we do the prebuilding ourselves, currently publishing to GitHub Releases
3. most package manager tools dynamically installable at run-time, from their origin internet sources

So in terms of research, it's a matter of working out which falls in (2) and which in (3).

Pre-built:

• node?
• php
• python
• ruby

Dynamically-installable:

• ?

(let's update the above lists as we go)

@viceice any good way to build up a definitive list - everything here? https://github.com/containerbase/buildpack/tree/main/src/usr/local/buildpack/tools

[viceice]

No easy way beside to check every tool.
I think most tools can be installed as runtime if we use a file download.

The morst rework is needed for erlang and java, as they are installed via apt.
For java i already have a solution in mind.
erlang needs some research

[rarkins]

Portable java seems possible via https://adoptopenjdk.net/releases.html?variant=openjdk11&jvmVariant=hotspot

Edit: Seems they changed project name and now have binaries in GitHub releases: https://github.com/adoptium?q=binaries&type=source&language=&sort=

[rarkins]

I think erlang is still pretty niche and I could live with it being unsupported for this for a while

[viceice]

Portable java seems possible via https://adoptopenjdk.net/releases.html?variant=openjdk11&jvmVariant=hotspot

Edit: Seems they changed project name and now have binaries in GitHub releases: https://github.com/adoptium?q=binaries&type=source&language=&sort=

I know, Some issues about adopt / adoptium:

• weird versioning scheme: jdk-11.0.9+11, jdk-11.0.9+11.1, jdk-11.0.9.1+1, jdk-11.0.10+9, jdk-11.0.11+9 . JDK8 has a even more weired versioning scheme
• then new adoptium don't provide older releases, so they only have v8, v11 and v16+

So i'm thinking if we should add a adoptium-java datasource and versioning to renovate, so we can directly use the binary releaes / adoptium api?

[viceice]

Maybe we should use the api at https://api.adoptium.net/ to get required version(s)

[viceice]

OK, if we use the api, we can use semver, at the api has a semver v2 field.

https://api.adoptopenjdk.net/v3/info/release_versions?architecture=x64&heap_size=normal&image_type=jdk&jvm_impl=hotspot&os=linux&page=0&page_size=10&project=jdk&release_type=ga&sort_method=DEFAULT&sort_order=ASC&vendor=adoptopenjdk

{
  "versions": [
    {
      "build": 13,
      "major": 8,
      "minor": 0,
      "openjdk_version": "8u181-b13",
      "security": 181,
      "semver": "8.0.181+13"
    },
    {
      "build": 12,
      "major": 8,
      "minor": 0,
      "openjdk_version": "8u192-b12",
      "security": 192,
      "semver": "8.0.192+12"
    },
    {
      "build": 8,
      "major": 8,
      "minor": 0,
      "openjdk_version": "8u202-b08",
      "security": 202,
      "semver": "8.0.202+8"
    },
    {
      "build": 3,
      "major": 8,
      "minor": 0,
      "openjdk_version": "8u212-b03",
      "security": 212,
      "semver": "8.0.212+3"
    },
    {
      "build": 4,
      "major": 8,
      "minor": 0,
      "openjdk_version": "8u212-b04",
      "security": 212,
      "semver": "8.0.212+4"
    },
    {
      "adopt_build_number": 1,
      "build": 10,
      "major": 8,
      "minor": 0,
      "openjdk_version": "1.8.0_222-b10",
      "security": 222,
      "semver": "8.0.222+10.1"
    },
    {
      "adopt_build_number": 1,
      "build": 9,
      "major": 8,
      "minor": 0,
      "openjdk_version": "1.8.0_232-b09",
      "security": 232,
      "semver": "8.0.232+9.1"
    },
    {
      "adopt_build_number": 1,
      "build": 8,
      "major": 8,
      "minor": 0,
      "openjdk_version": "1.8.0_242-b08",
      "security": 242,
      "semver": "8.0.242+8.1"
    },
    {
      "adopt_build_number": 1,
      "build": 9,
      "major": 8,
      "minor": 0,
      "openjdk_version": "1.8.0_252-b09",
      "security": 252,
      "semver": "8.0.252+9.1"
    },
    {
      "build": 10,
      "major": 8,
      "minor": 0,
      "openjdk_version": "1.8.0_262-b10",
      "security": 262,
      "semver": "8.0.262+10"
    }
  ]
}