Auto-publish Arm64 images when emscripten-releases-tags.json is updated#1231
Conversation
| name: push image | ||
| command: | | ||
| docker login -u "$DOCKER_USER" -p "$DOCKER_PASS" | ||
| make -C ./docker version=${ARM64_VERSION} alias=${ARM64_IMAGE_TAG} only_alias=true push |
There was a problem hiding this comment.
What would the preferred tag be for these images? This PR would currently set it to <emscripten-version>-arm64 e.g. 3.1.33-arm64
There was a problem hiding this comment.
Does docker have a convention for this? Is the architecture not an inherent part of the image? What happens if you try to install the normal x64 imagine and arm host?
There was a problem hiding this comment.
Great questions and thanks for the prompt reply!
Does docker have a convention for this? Is the architecture not an inherent part of the image?
Docker does offer an experimental solution with Docker Manifest.
The official docker guide would suggest tagging your images with an arch suffix e.g. build each arch image separately and tag with 3.1.33-amd64 and 3.1.33-arm64v8. Then with the docker manifest tool you can alias these tags under 3.1.33 and push to the docker repo. When a consumer pulls an image like emscripten/emsdk:3.1.33, docker can auto select the appropriate arch for the host machine.
Moving to the manifest solution will increase the complexity of the current CI docker steps. Perhaps something that could be followed up with in the future?
What happens if you try to install the normal x64 image and arm host?
Let me know if I've misunderstood. If a user installed both an amd64 and arm64v8 images they would have different docker tags so there's no conflict. Consumers would have to explicitly opt-in to using ARM based images by using the special suffixed tag (emscripten/emsdk:<version>-arm64v8), unless the manifest solution above was implemented.
Edit: Ah i think you're talking about installing and running x64 images on an Arm host. This would be the same experience any docker user currently experiences when running an image architecture different from the host machine.
|
Just checking, is this the issue to subscribe to observe the progress on latest emsdks being available for dockers? - Issue reported on emscripten-core/emscripten#19275 ? |
Specifically, this issue related to publish arm64 versions of the docker image. The x86_64 images are already auto-published continuously. |
jamsinclair
force-pushed
the
automated_docker_arm64_publish
branch
from
7044f0f to
98feb80
Compare
|
Hey @sbc100 is there anything you would like to see to make this PR a possibility? 🤓 As it stands, this pull request would automatically publish a special image tag, I think the simplest long term solution would be to release multiarch docker images. This can be conveniently done in one command with BuildKit, |
| name: extract latest emscripten arm64 linux version | ||
| command: | | ||
| export ARM64_VERSION=$(grep -oP '(?<="latest-arm64-linux": ")[^"]*' emscripten-releases-tags.json) | ||
| export ARM64_IMAGE_TAG=${ARM64_VERSION}-arm64 |
There was a problem hiding this comment.
To these exported variables persist to the next run ? My recollection was that they do not, and that you need to persist them somehow/somewhere?
There was a problem hiding this comment.
You are correct! Thanks for pointing that out. I've updated it so that it exports correctly between commands
jamsinclair
left a comment
jamsinclair
left a comment
There was a problem hiding this comment.
Thanks for looking at the changes so swiftly @sbc100 – apologies I wasn't as quick to get back to your comments.
I've simulated the publish job on my own CircleCI account to ensure that this job works as expected. You can see the steps running at https://app.circleci.com/pipelines/github/jamsinclair/emsdk-circleci-test/8/workflows/8dffc43c-ef41-43b5-a973-c8778255612d/jobs/8
Feel free to tag me if there's any more changes required or ongoing support needed, thanks 🙇
| linux_arm64: | ||
| machine: | ||
| image: ubuntu-2004:202101-01 | ||
| image: ubuntu-2004:2023.07.1 |
There was a problem hiding this comment.
I just pulled the most recent-ish release from https://circleci.com/docs/using-arm/#images-with-arm-support.
Please let me know if there's a need to stay on 202101-01?
…ed (emscripten-core#1231) Add job to conditionally build arm64 images when emscripten-releases-tags.json updated
3 participants
Fixes: #1206 #1211
Hi Maintainers, firstly thank you for all your work around the emscripten project!
I would like to propose this PR as a potential solution to some issues identified in #1206 and #1211
emscripten-releases-tags.jsonfile has changed and there isn't already a published version for the currentlatest-arm64-linuxversionI understand if this is not an extra complexity you want to support for the project yet, if so feel free to close the PR 🙇