Skip to content

Conversation

@tianon
Copy link
Contributor

@tianon tianon commented Feb 20, 2019

The contents of these files were generated by cross-referencing the available artifacts on https://nodejs.org/dist/ vs the supported architectures of the base image of each variant.

Closes #985

I would again echo my distaste for the overly-complex and confusing structure of these files that I articulated in #863. IMO it would much cleaner and simpler to maintain a list of mappings from "bashbrew architecture" string to "node architecture" string (which is a trivial list of 7 simple mappings) and generate the rest of the missing information based on detection (checking the release folder and then cross-referencing it with the supported architectures of the base image, which is exactly what my script did to generate these based on that list of 7 mappings).

Here's a useful diff to more easily see the actual effect of this change (comparing the downstream effect):

$ diff -u <(bashbrew cat https://github.com/docker-library/official-images/raw/master/library/node) <(bashbrew cat <(./generate-stackbrew-library.sh))
--- /dev/fd/63	2019-02-19 16:36:39.546281014 -0800
+++ /dev/fd/62	2019-02-19 16:36:39.550280903 -0800
@@ -42,11 +42,12 @@
 Directory: 6/jessie-slim
 
 Tags: 6.16.0-alpine, 6.16-alpine, 6-alpine, boron-alpine
+Architectures: amd64, arm32v6, arm64v8, i386, ppc64le, s390x
 GitCommit: daa131d713cf42ae181292471766879f750b5230
 Directory: 6/alpine
 
 Tags: 6.16.0-onbuild, 6.16-onbuild, 6-onbuild, boron-onbuild
-Architectures: amd64, arm32v7, i386
+Architectures: amd64, arm32v7, arm64v8, i386, ppc64le, s390x
 GitCommit: daa131d713cf42ae181292471766879f750b5230
 Directory: 6/onbuild
 
@@ -56,12 +57,12 @@
 Directory: 6/stretch
 
 Tags: 6.16.0-stretch-slim, 6.16-stretch-slim, 6-stretch-slim, boron-stretch-slim, 6.16.0-slim, 6.16-slim, 6-slim, boron-slim
-Architectures: amd64, arm32v7, i386
+Architectures: amd64, arm32v7, arm64v8, i386, ppc64le, s390x
 GitCommit: daa131d713cf42ae181292471766879f750b5230
 Directory: 6/stretch-slim
 
 Tags: 11.10.0-alpine, 11.10-alpine, 11-alpine, current-alpine, alpine
-Architectures: amd64, arm32v6, arm64v8, i386, ppc64le, s390x
+Architectures: amd64, arm32v6, arm64v8, ppc64le, s390x
 GitCommit: 7b643da4aeb3d77f1c26c9f6fadd32b04f8d7bc9
 Directory: 11/alpine
 
@@ -86,7 +87,7 @@
 Directory: 10/jessie-slim
 
 Tags: 10.15.1-alpine, 10.15-alpine, 10-alpine, dubnium-alpine, lts-alpine
-Architectures: amd64, arm32v6, arm64v8, i386, ppc64le, s390x
+Architectures: amd64, arm32v6, arm64v8, ppc64le, s390x
 GitCommit: 90043cdde5057865b94fec447ce193fb46b69e18
 Directory: 10/alpine
 
@@ -96,7 +97,7 @@
 Directory: 10/stretch
 
 Tags: 10.15.1-stretch-slim, 10.15-stretch-slim, 10-stretch-slim, dubnium-stretch-slim, lts-stretch-slim, 10.15.1-slim, 10.15-slim, 10-slim, dubnium-slim, lts-slim
-Architectures: amd64, arm32v7
+Architectures: amd64, arm32v7, arm64v8, ppc64le, s390x
 GitCommit: 90043cdde5057865b94fec447ce193fb46b69e18
 Directory: 10/stretch-slim
 

I'm happy to share the script I wrote to generate these with appropriate values, but I'd honestly really prefer if this could be solved in a way in this repository directly that makes these files actually maintainable without doing this full hacky generation like what I've done here.

If you want a good example of a fairly-static list of supported architectures per release version, see https://github.com/docker-library/golang (especially the release-architectures files in each version folder).

If you want an example of a dynamic list of supported architectures per release version (where architectures can shift to come and go), see https://github.com/docker-library/pypy instead.

@tianon
Copy link
Contributor Author

tianon commented Feb 20, 2019

This is technically slightly wrong -- there should be Alpine for all the i386 variants since it's building from source; give me a minute to update my script to handle that case too and regenerate real fast.

The contents of these files were generated by cross-referencing the available artifacts on https://nodejs.org/dist/ vs the supported architectures of the base image of each variant.
@tianon
Copy link
Contributor Author

tianon commented Feb 20, 2019

Updated downstream diff:

$ diff -u <(bashbrew cat https://github.com/docker-library/official-images/raw/master/library/node) <(bashbrew cat <(./generate-stackbrew-library.sh))
--- /dev/fd/63	2019-02-19 16:47:06.148765005 -0800
+++ /dev/fd/62	2019-02-19 16:47:06.152764891 -0800
@@ -42,11 +42,12 @@
 Directory: 6/jessie-slim
 
 Tags: 6.16.0-alpine, 6.16-alpine, 6-alpine, boron-alpine
+Architectures: amd64, arm32v6, arm64v8, i386, ppc64le, s390x
 GitCommit: daa131d713cf42ae181292471766879f750b5230
 Directory: 6/alpine
 
 Tags: 6.16.0-onbuild, 6.16-onbuild, 6-onbuild, boron-onbuild
-Architectures: amd64, arm32v7, i386
+Architectures: amd64, arm32v7, arm64v8, i386, ppc64le, s390x
 GitCommit: daa131d713cf42ae181292471766879f750b5230
 Directory: 6/onbuild
 
@@ -56,7 +57,7 @@
 Directory: 6/stretch
 
 Tags: 6.16.0-stretch-slim, 6.16-stretch-slim, 6-stretch-slim, boron-stretch-slim, 6.16.0-slim, 6.16-slim, 6-slim, boron-slim
-Architectures: amd64, arm32v7, i386
+Architectures: amd64, arm32v7, arm64v8, i386, ppc64le, s390x
 GitCommit: daa131d713cf42ae181292471766879f750b5230
 Directory: 6/stretch-slim
 
@@ -96,7 +97,7 @@
 Directory: 10/stretch
 
 Tags: 10.15.1-stretch-slim, 10.15-stretch-slim, 10-stretch-slim, dubnium-stretch-slim, lts-stretch-slim, 10.15.1-slim, 10.15-slim, 10-slim, dubnium-slim, lts-slim
-Architectures: amd64, arm32v7
+Architectures: amd64, arm32v7, arm64v8, ppc64le, s390x
 GitCommit: 90043cdde5057865b94fec447ce193fb46b69e18
 Directory: 10/stretch-slim
 

amd64 jessie,jessie-slim,alpine,onbuild,stretch,stretch-slim
arm32v7 jessie,jessie-slim,stretch,stretch-slim
arm64v8 alpine,stretch,stretch-slim
i386 alpine
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this one different than all the other version?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you check out https://nodejs.org/dist/latest-v10.x/, there's no x86 artifact anymore (#823), so only Alpine is supported there (since it builds from source).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, looking at the output of this:

 curl -Ss https://nodejs.org/dist/index.json | json -c 'this.version.includes("v10")' | json -ag version files

I don't see it either for v10

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, good to know!!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to make sure, @MylesBorins @rvagg is this intentional?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't even test Node on x86 Linux any more, it's marked "experimental" which means that our infra doesn't test for it, we don't build it but we'll accept community contributions for it. That leaves downstream packagers in an awkward position because you get less support.

I'd suggest that you should drop x86 from Docker. But it'd also be interesting to know what the x86 Docker userbase looks like. Do you have metrics that would indicate what the impact would be?

On another note, I'm pondering setting up an "alternative builds" project that's a little separate to the official Node build infrastructure and nodejs.org offerings. There's a few things that could be offloaded to it, Linux x86 and ARMv6 being the obvious two and Linux musl being a possible addition. We're trying hard to simplify our infra on the official Node end because it's been getting out of hand so setting up something on the side that's unofficial and maybe maintained by an entirely different set of people would be a positive step forward.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

^ I meant to add that I'd like to gauge interest in such a project and whether there'd be people here willing to contribute time and maybe organisations that might be willing to contribute $$ if needed.

Copy link

@refack refack Mar 2, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm working on an "alternative" dist of linux/ia32 for the Internet Archive people. The idea is to have a best-effort-supported dist, and stop when it breaks or becomes intractable.
BTW their use case is embedded systems, so it's not obvious to me if that correlates with Docker use cases.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should I update this to drop architectures that Node.js does not publish binary artifacts for?

I assume that means "no binaries at all", there would still be binaries for Alpine even though we don't ship binaries for that on Nodejs.org, Right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I simply mean dropping things like x86/i386 which are no longer built for officially (even though Alpine images are built from source and could thus still be provided with 32bit builds).

@PeterDaveHello PeterDaveHello requested a review from a team February 22, 2019 15:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants