Skip to content

USHIFT-247: Generate release-info #1196

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
openshift-merge-robot merged 2 commits into from
Dec 16, 2022
Merged

USHIFT-247: Generate release-info #1196

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
c109bf5
Add release-info package
fzdarsky Dec 15, 2022
8fc3c1d
Add offline container doc
fzdarsky Dec 16, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ export BIN_TIMESTAMP ?=$(shell date -u +'%Y-%m-%dT%H:%M:%SZ')
export TIMESTAMP ?=$(shell echo $(BIN_TIMESTAMP) | tr -d ':' | tr 'T' '-' | tr -d 'Z')
SOURCE_GIT_COMMIT_TIMESTAMP ?= $(shell TZ=UTC0 git show --quiet --date='format-local:%Y%m%d%H%M%S' --format="%cd")

OCP_VERSION := $(shell awk -F'["-]' '/var Base/ {print $$2}' ${PROJECT_DIR}/pkg/release/release.go)
OCP_VERSION := $(shell jq -r '.release.base' ${PROJECT_DIR}/assets/release/release-$(shell uname -i).json)
MICROSHIFT_VERSION ?= $(subst -clean,,$(shell echo '${OCP_VERSION}-${SOURCE_GIT_COMMIT_TIMESTAMP}-${SOURCE_GIT_COMMIT}-${SOURCE_GIT_TREE_STATE}'))

# Overload SOURCE_GIT_TAG value set in vendor/github.com/openshift/build-machinery-go/make/lib/golang.mk
Expand Down
1 change: 1 addition & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ To install, configure and run MicroShift, refer to the following documentation:

- [Getting Started with MicroShift](./docs/getting_started.md)
- [MicroShift Configuration](./docs/howto_config.md)
- [Embeddding MicroShift's containers for offline deployments](./docs/howto_offline_containers.md)
- [MicroShift Behind Proxy](./docs/howto_http_proxy.md)
- [Load Balancer for User Workloads](./docs/howto_load_balancer.md)
- [AMQ Broker on MicroShift](./docs/howto_amq_broker.md)
Expand Down
2 changes: 1 addition & 1 deletion assets/embed.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ package embedded

import "embed"

//go:embed components controllers core crd version
//go:embed components controllers core crd version release
var content embed.FS

func Asset(name string) ([]byte, error) {
Expand Down
20 changes: 20 additions & 0 deletions assets/release/release-aarch64.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
{
"release": {
"base": "4.12.0-0.nightly-arm64-2022-12-15-220457"
},
"images": {
"cli": "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:caa0fe9b53e4b0d2c9988fc20fa745ad80817a7cec36ba098d1812fab03e2add",
"coredns": "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cbe32c3940f369eb9093d2b6669a22ce4fd3b1c0781c2afd74f1b0b1e6bd3a9d",
"haproxy-router": "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e19d3bcfc797cd879db6043ef59c1d9e9c8c199181f5f2b6cca5e4c7cfed0a5d",
"kube-rbac-proxy": "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:329566d40a19ff6914c4d584c7526c2093917a1437eb32c9f299f1c62350d035",
"odf-topolvm": "quay.io/rh-storage-partners/microshift-topolvm@sha256:616fe64c9f2d1315cec655d482e7b26596594e879e07017e0e610d37c72bacd0",
"openssl": "registry.access.redhat.com/ubi8/openssl@sha256:9e743d947be073808f7f1750a791a3dbd81e694e37161e8c6c6057c2c342d671",
"csi-external-provisioner": "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f8a246885c509a113cbd7ce43f78ea764752fad2f1bf2b61849abcaa77baacff",
"csi-external-resizer": "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b9524eb63c3408c2889ec926f2ebdf9d4ab4689c3ad50594eb8d80a9bdd0dbc9",
"csi-node-driver-registrar": "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:66d99027850fac4ed6f6f9cef8f6247c08881f75648492b76a9e7f50ff9dc115",
"csi-livenessprobe": "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ea75863f09d2a45ef549e08b6c75fe2058ca142ecc53d793408d5a80982c90d7",
"ovn-kubernetes-microshift": "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:1f114ebd882e37adeeea576de1d73f64eec0696c0950f8b70434a1a2c68433ed",
"pod": "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:506004700e9de44e72552ce080113563a50c00e90289cbec2ca45d8eb5580b7b",
"service-ca-operator": "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2b601e2889b25a6175c9d65c834c310262696094a9bcb5f49d6c4e2682392727"
}
}
20 changes: 20 additions & 0 deletions assets/release/release-x86_64.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
{
"release": {
"base": "4.12.0-0.nightly-2022-12-15-175145"
},
"images": {
"cli": "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9945c3f5475a37e145160d2fe6bb21948f1024a856827bc9e7d5bc882f44a750",
"coredns": "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:82cfef91557f9a70cff5a90accba45841a37524e9b93f98a97b20f6b2b69e5db",
"haproxy-router": "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a66f5038b4499e3c069067365c8426388d09bf3ac4dd8eb8bcbd39cd5f6c6ed0",
"kube-rbac-proxy": "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:dac19e13041b0bb5d60ecaab219a43c2b4ea57a082cf13bc6305acf86254432e",
"odf-topolvm": "quay.io/rh-storage-partners/microshift-topolvm@sha256:616fe64c9f2d1315cec655d482e7b26596594e879e07017e0e610d37c72bacd0",
"openssl": "registry.access.redhat.com/ubi8/openssl@sha256:9e743d947be073808f7f1750a791a3dbd81e694e37161e8c6c6057c2c342d671",
"csi-external-provisioner": "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:82983bdae16b3cadc78539f687ea39f6bb7af1ed99f3382fdbcb61500ed30398",
"csi-external-resizer": "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e2b951d8b5f88142bc1a7f5ca8529e3a6a89c8ff2dd78c9c06e8f6194e3d681f",
"csi-node-driver-registrar": "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f8f85305bd4a9fc8796a05cb27676e084e80562c8af421fa99b44ef0441beff9",
"csi-livenessprobe": "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:46bfcb0440620f12a91be6493039b07bf7c96d104a1a59b3e8ed3caaec2dda5c",
"ovn-kubernetes-microshift": "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:10ad432ad8c4c3e6d61dfb63e1f5304e6d87c5f5be67d332c82e5364880838b7",
"pod": "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:32d435c9ff716e2d7e123f9b16f17059356ce8cceb7f10a95bfa99373b844e89",
"service-ca-operator": "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:20fd4b23b852db6f9f0d0134ad8d9128cb771af16723a8e239bcd97e5cd874b4"
}
}
9 changes: 5 additions & 4 deletions docs/devenv_rhel8.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -91,10 +91,11 @@ make srpm
The artifacts of the build are located in the `_output/rpmbuild` directory.
```bash
$ cd ~/microshift/ && find _output -name \*.rpm
_output/rpmbuild/RPMS/x86_64/microshift-4.12.0-2022_10_30_114426_untagged.el8.x86_64.rpm
_output/rpmbuild/RPMS/x86_64/microshift-networking-4.12.0-2022_10_30_114426_untagged.el8.x86_64.rpm
_output/rpmbuild/RPMS/noarch/microshift-selinux-4.12.0-2022_10_30_114426_untagged.el8.noarch.rpm
_output/rpmbuild/SRPMS/microshift-4.12.0-2022_10_30_114426_untagged.el8.src.rpm
_output/rpmbuild/RPMS/x86_64/microshift-4.12.0_20221215165847_758c8b61-1.el8.x86_64.rpm
_output/rpmbuild/RPMS/x86_64/microshift-networking-4.12.0_20221215165847_758c8b61-1.el8.x86_64.rpm
_output/rpmbuild/RPMS/noarch/microshift-release-info-4.12.0_20221215165847_758c8b61-1.el8.noarch.rpm
_output/rpmbuild/RPMS/noarch/microshift-selinux-4.12.0_20221215165847_758c8b61-1.el8.noarch.rpm
_output/rpmbuild/SRPMS/microshift-4.12.0_20221215165847_758c8b61-1.el8.src.rpm
```

## Run MicroShift Executable
Expand Down
57 changes: 57 additions & 0 deletions docs/howto_offline_containers.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
# Embedding MicroShift's container iamges for offline deployments

Image Builder supports building `rpm-ostree` system images with embedded container images. Embedded container images are immediately available to container engines like `podman` and `cri-o` after booting the system, without having to pull images over the network from a container registry. This means container workloads can start up without delay and without requiring network connectivity.

To embed a container image, add it to the Image Builder blueprint as follows:

```
[[containers]]
source = "<your_image_pullspec_with_tag_or_digest>"
```

To embed MicroShift's container images, you need to know the exact list of container image references used by the MicroShift version you're deploying. You can obtain this list by installing the `microshift-release-info` RPM package of the same version, for example:

```
$ sudo dnf install -y microshift-release-info-4.12.0-1
$ ls /usr/share/microshift/release
release-aarch64.json release-x86_64.json
```

Alternatively, you can download and unpack the RPM package without installing it:

```
$ sudo dnf download microshift-release-info-4.12.0-1
microshift-release-info-4.12.0-1.el8.noarch.rpm
$ rpm2cpio microshift-release-info-4.12.0-1.el8.noarch.rpm | cpio -idmv
./usr/share/microshift/release/release-aarch64.json
./usr/share/microshift/release/release-x86_64.json
```

Using the release info for your CPU architecture, you can now generate the section to embed the container images to your `blueprint.toml`:

```
$ jq -r '.images | .[] | ("[[containers]]\nsource = \"" + . + "\"\n")' release-$(uname -i).json >> blueprint.tonl
```

Remember to pin the version of the MicroShift RPMs in the blueprint to the version matching your container images. The resulting `blueprint.toml` should look simiar to this:

```
name = "microshift-offline"

description = ""
version = "0.0.1"
modules = []
groups = []

[[packages]]
name = "microshift"
version = "4.12.0-1"

[[containers]]
source = "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9945c3f5475a37e145160d2fe6bb21948f1024a856827bc9e7d5bc882f44a750"

[[containers]]
source = "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:82cfef91557f9a70cff5a90accba45841a37524e9b93f98a97b20f6b2b69e5db"

...
```
7 changes: 3 additions & 4 deletions docs/rhel4edge_iso.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,8 +55,7 @@ Optional arguments:
Path to one or more comma-separated RPM packages to be
included in the image (default: none)
-embed_containers
Embed the MicroShift container dependencies in the image using the
'pkg/release/get.sh images $(uname -i)' command to get their list
Embed the MicroShift container dependencies in the image
-ostree_server_name name_or_ip
Name or IP address and optionally port of the ostree
server (default: 127.0.0.1:8080)
Expand Down Expand Up @@ -153,7 +152,7 @@ Proceed by running the build script with the `-embed_containers` argument to inc

When executed in this mode, the `scripts/image-builder/build.sh` script performs an extra step to append the list of the MicroShift container images to the blueprint so that they are installed when the operating system boots for the first time. The list of these images can be obtained by the following command.
```bash
~/microshift/pkg/release/get.sh images $(uname -i)
jq -r '.images | .[]' ~/microshift/assets/release/release-$(uname -i).json
```

## Install MicroShift for Edge
Expand All @@ -167,7 +166,7 @@ sudo scp microshift@microshift-dev:/home/microshift/microshift/_output/image-bui
Run the following commands to create a virtual machine using the installer image.
```bash
VMNAME="microshift-edge"
VERSION=$(~/microshift/pkg/release/get.sh base)
VERSION=$(grep "^var Base =" ~/microshift/pkg/release/release.go | cut -d\" -f 2)
sudo -b bash -c " \
cd /var/lib/libvirt/images/ && \
virt-install \
Expand Down
29 changes: 26 additions & 3 deletions packaging/rpm/microshift.spec
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,18 @@ Requires: sos
%{?systemd_requires}

%description
The MicroShift package provides an OpenShift Kubernetes distribution optimized for small form factor and edge computing.
The microshift package provides an OpenShift Kubernetes distribution optimized for small form factor and edge computing.
Comment thread
fzdarsky marked this conversation as resolved.
Outdated


%package release-info
Summary: Release information for MicroShift
BuildArch: noarch

%description release-info
The microshift-release package provides release information files for this
release. These files contain the list of container image references used by
MicroShift and can be used to embed those images into osbuilder blueprints.


%package selinux
Summary: SELinux policies for MicroShift
Expand All @@ -72,7 +83,8 @@ BuildArch: noarch
%{?selinux_requires}

%description selinux
The MicroShift SELinux package provides the SELinux policy modules required by MicroShift.
The microshift-selinux package provides the SELinux policy modules required by MicroShift.


%package networking
Summary: Networking components for MicroShift
Expand All @@ -82,7 +94,8 @@ Requires: NetworkManager-ovs
Requires: jq

%description networking
The MicroShift Networking package provides the networking components necessary for the MicroShift default CNI driver.
The microshift-networking package provides the networking components necessary for the MicroShift default CNI driver.


%prep

Expand Down Expand Up @@ -143,6 +156,10 @@ install -p -m644 packaging/systemd/microshift.service %{buildroot}%{_unitdir}/mi
install -d -m755 %{buildroot}/%{_sysconfdir}/microshift
install -p -m644 packaging/microshift/config.yaml %{buildroot}%{_sysconfdir}/microshift/config.yaml.default

# release-info files
mkdir -p -m755 %{buildroot}%{_datadir}/microshift/release
install -p -m644 assets/release/release*.json %{buildroot}%{_datadir}/microshift/release

# Memory tweaks to the OpenvSwitch services
mkdir -p -m755 %{buildroot}%{_sysconfdir}/systemd/system/ovs-vswitchd.service.d
mkdir -p -m755 %{buildroot}%{_sysconfdir}/systemd/system/ovsdb-server.service.d
Expand Down Expand Up @@ -212,6 +229,9 @@ systemctl enable --now --quiet openvswitch || true
%{_sysconfdir}/crio/crio.conf.d/microshift.conf
%config(noreplace) %{_sysconfdir}/microshift/config.yaml.default

%files release-info
%{_datadir}/microshift/release/release*.json

%files selinux

/var/run/kubelet
Expand All @@ -237,6 +257,9 @@ systemctl enable --now --quiet openvswitch || true
# Use Git command to generate the log and replace the VERSION string
# LANG=C git log --date="format:%a %b %d %Y" --pretty="tformat:* %cd %an <%ae> VERSION%n- %s%n" packaging/rpm/microshift.spec
%changelog
* Wed Dec 14 2022 Frank A. Zdarsky <fzdarsky@redhat.com> 4.12.0
- Add microshift-release-info subpackage

* Wed Dec 07 2022 Gregory Giguashvili <ggiguash@redhat.com> 4.12.0
- Update the summaries and descriptions of MicroShift RPM packages

Expand Down
45 changes: 0 additions & 45 deletions pkg/release/get.sh
View file
Open in desktop

This file was deleted.

56 changes: 40 additions & 16 deletions pkg/release/release.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,20 +16,44 @@ limitations under the License.

package release

var Base = "4.12.0-0.nightly-2022-12-15-175145"

var Image = map[string]string{
"cli": "quay.io/microshift/cli:" + Base,
"coredns": "quay.io/microshift/coredns:" + Base,
"haproxy_router": "quay.io/microshift/haproxy-router:" + Base,
"kube_rbac_proxy": "quay.io/microshift/kube-rbac-proxy:" + Base,
"odf_topolvm": "quay.io/microshift/odf-topolvm-rhel8" + Base,
"openssl": "quay.io/microshift/openssl" + Base,
"csi_external_provisioner": "quay.io/microshift/csi-external-provisioner" + Base,
"csi_external_resizer": "quay.io/microshift/csi-external-resizer" + Base,
"csi_node_driver_registrar": "quay.io/microshift/csi-node-driver-registrar" + Base,
"csi_livenessprobe": "quay.io/microshift/csi-livenessprobe" + Base,
"ovn_kubernetes_microshift": "quay.io/microshift/ovn-kubernetes-microshift:" + Base,
"pod": "quay.io/microshift/pause:" + Base,
"service_ca_operator": "quay.io/microshift/service-ca-operator:" + Base,
import (
"encoding/json"
"fmt"
"runtime"
"strings"

embedded "github.com/openshift/microshift/assets"
)

var Base = "undefined"

var Image = map[string]string{}

func init() {
arch_replacer := strings.NewReplacer("amd64", "x86_64", "arm64", "aarch64")
Copy link
Copy Markdown
Contributor

@oglok oglok Dec 15, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder why arch names have been replaced: amd64 to x86_64 and arm64 to aarch64. Any specific reason?

Copy link
Copy Markdown
Contributor Author

@fzdarsky fzdarsky Dec 15, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My motivation for this was user experience. The x86_64/aarch64 architecture names are more idiomatic on Linux systems in general and RPM-based distros in particular.

For example, this way users can use $(uname -i) in scripts and avoid conditional logic and substitutions as with the Golang names. It’s also consistent with the RPM package naming scheme.

arch := arch_replacer.Replace(runtime.GOARCH)

release_file := "release/release-" + arch + ".json"
Copy link
Copy Markdown
Contributor

@mangelajo mangelajo Dec 16, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be great if we could also produce a toml snippet so customers just can append that to their blueprints without additional processing if they wanted.

I suggest that as follow up, not now.

The toml is less descriptive for the name of the images, but we could put that in comments.

data, err := embedded.Asset(release_file)
if err != nil {
// If there is no release file for this architecture, work with the generic specs
Comment thread
fzdarsky marked this conversation as resolved.
Outdated
return
}

var release map[string]any
if err := json.Unmarshal(data, &release); err != nil {
panic(fmt.Errorf("unmarshaling %s: %v", release_file, err))
}

// Copy in the OCP base version
metadata := release["release"].(map[string]any)
Base = metadata["base"].(string)

// Copy in the pullspecs, translating the keys as used by the OCP release image
// (with '-'s) into keys we can use in go templates (need to use '_'s instead).
images := release["images"].(map[string]any)
for name, pullspec := range images {
name := strings.Replace(name, "-", "_", -1)
Image[name] = pullspec.(string)
}
Comment thread
fzdarsky marked this conversation as resolved.
Outdated
}
38 changes: 0 additions & 38 deletions pkg/release/release_amd64.go
View file
Open in desktop

This file was deleted.

Loading