test: migrate pytest-operator to jubilant

.github/workflows/integration_test.yaml

@@ -3,20 +3,18 @@ name: Integration tests
 on:
   pull_request:
   schedule:
-  - cron: "0 15 * * SAT"
+    - cron: "0 15 * * SAT"
 
 concurrency:
-  group: ${{ github.workflow }}-${{ github.event.pull_request.number || 
-    github.ref }}
+  group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
   cancel-in-progress: true
 
 jobs:
   integration-tests:
-    uses: 
-      canonical/operator-workflows/.github/workflows/integration_test.yaml@main
+    uses: canonical/operator-workflows/.github/workflows/integration_test.yaml@main
     secrets: inherit
     with:
-      juju-channel: 3.6/stable
+      juju-channel: 4/stable
       provider: lxd
       modules: '["test_charm", "test_upgrade"]'
       self-hosted-runner: true
@@ -28,5 +26,5 @@ jobs:
   allure-report:
     if: ${{ !cancelled() && github.event_name == 'schedule' }}
     needs:
-    - integration-tests
+      - integration-tests
     uses: canonical/operator-workflows/.github/workflows/allure_report.yaml@main

.vale/styles/config/vocabularies/local/accept.txt

@@ -1,3 +1,5 @@
+aproxy
 chroot
 cron
+opentelemetry
 pipx

CONTRIBUTING.md

@@ -27,8 +27,11 @@ tox                      # runs 'format', 'lint', and 'unit' environments
 
 
 The integration tests (both of the charm and the app)
-require options to be passed via the command line (see `tests/conftest.py`) and 
-environment variables `OPENSTACK_PASSWORD` to be able to deploy the charm and/or upload images to OpenStack.
+require options to be passed through the command line (see `tests/conftest.py`) and
+architecture-specific OpenStack password environment variables, for example
+`OPENSTACK_PASSWORD_AMD64`, to deploy the charm and/or upload images to OpenStack.
+If you are testing multiple architectures, set the corresponding `OPENSTACK_PASSWORD_<ARCH>`
+variables for each one. The tests create and use Juju secrets from these values during setup.
 
 ## Build the charm
 

README.md

@@ -1,33 +1,39 @@
 <!-- vale Canonical.007-Headings-sentence-case = NO -->
+
 # GitHub runner image builder operator
+
 <!-- vale Canonical.007-Headings-sentence-case = YES -->
 <!-- Use this space for badges -->
 
-A Juju charm that provides the GitHub runner workload embedded snapshot image to the 
+A Juju charm that provides the GitHub runner workload embedded snapshot image to the
 [GitHub runner](https://charmhub.io/github-runner) charm. This charm is deployed as a VM and works
 on top of OpenStack infrastructure.
 
 Like any Juju charm, this charm supports one-line deployment, configuration, integration, scaling,
 and more. For Charmed GitHub runner image builder, this includes support for configuring:
-* Multi-arch
-* Multi Ubuntu bases
-* Juju/MicroK8s snap channels
-* External scripts
 
-For information about how to deploy, integrate, and manage this charm, see the Official 
+- Multi-arch
+- Multi Ubuntu bases
+- Juju/MicroK8s snap channels
+- External scripts
+
+For information about how to deploy, integrate, and manage this charm, see the Official
 [CharmHub Documentation](https://charmhub.io/github-runner-image-builder).
 
 ## Get started
+
 <!--Briefly summarize what the user will achieve in this guide.-->
+
 Deploy GitHub runner image builder with GitHub runners.
 
 <!--Indicate software and hardware prerequisites-->
+
 You'll need a working [OpenStack installation](https://microstack.run/docs/single-node) with
 flavors with a minimum of 2 CPU cores, 8GB RAM and 10GB disk.
 
 ### Set up
 
-Follow [MicroStack's single-node](https://microstack.run/docs/single-node) starting guide to set 
+Follow [MicroStack's single-node](https://microstack.run/docs/single-node) starting guide to set
 up MicroStack.
 
 Follow the [tutorial on GitHub runner](https://charmhub.io/github-runner) to deploy the GitHub
@@ -38,30 +44,36 @@ runner.
 Deploy the charm.
 
 ```
+juju add-secret openstack-password password=<OPENSTACK-PASSWORD>
+OPENSTACK_PASSWORD_SECRET=$(juju show-secret openstack-password --format json | jq -r 'keys[0]')
+
 juju deploy github-runner-image-builder \
---config experimental-external-build=True \
---config experimental-external-build-network=<OPENSTACK-NETWORK-NAME> \
+--config build-network=<OPENSTACK-NETWORK-NAME> \
 --config openstack-auth-url=<OPENSTACK-AUTH-URL> \
---config openstack-password=<OPENSTACK-PASSWORD> \
+--config openstack-password-secret=$OPENSTACK_PASSWORD_SECRET \
 --config openstack-project-domain-name=<OPENSTACK-PROJECT-DOMAIN-NAME> \
 --config openstack-project-name=<OPENSTACK-PROJECT-NAME> \
+--config openstack-user-domain-name=<OPENSTACK-USER-DOMAIN-NAME> \
 --config openstack-user-name=<OPENSTACK-USER-NAME>
 
 juju integrate github-runner-image-builder github-runner
 ```
 
 ### Basic operations
+
 <!--Brief walkthrough of performing standard configurations or operations-->
 
 After having deployed and integrated the charm with the GitHub runner charm, the image should start
 to build and be provided to the GitHub runner automatically. The whole process takes around 10
 minutes.
 
 ## Integrations
-<!-- Information about particularly relevant interfaces, endpoints or libraries related to the charm. For example, peer relation endpoints required by other charms for integration.--> 
-* image: The image relation provides the OpenStack image ID to the GitHub runners.
-* cos-agent: The COS agent subordinate charm provides observability using the Canonical
-Observability Stack (COS).
+
+<!-- Information about particularly relevant interfaces, endpoints or libraries related to the charm. For example, peer relation endpoints required by other charms for integration.-->
+
+- image: The image relation provides the OpenStack image ID to the GitHub runners.
+- cos-agent: The COS agent subordinate charm provides observability using the Canonical
+  Observability Stack (COS).
 
 For a full list of integrations, please refer to the [Charmhub documentation](https://charmhub.io/github-runner-image-builder/integrations).
 
@@ -71,11 +83,13 @@ This repository contains the charm in the root directory and the `github-runner-
 application in the `app` directory. Refer to [Contributing](CONTRIBUTING.md) for more information.
 
 ## Learn more
-* [Read more](https://charmhub.io/github-runner-image-builder) <!--Link to the charm's official documentation-->
-* [Developer documentation](https://github.com/canonical/github-runner-image-builder-operator/blob/main/CONTRIBUTING.md) <!--Link to any developer documentation-->
-* [Troubleshooting](https://matrix.to/#/#charmhub-charmdev:ubuntu.com)
+
+- [Read more](https://charmhub.io/github-runner-image-builder) <!--Link to the charm's official documentation-->
+- [Developer documentation](https://github.com/canonical/github-runner-image-builder-operator/blob/main/CONTRIBUTING.md) <!--Link to any developer documentation-->
+- [Troubleshooting](https://matrix.to/#/#charmhub-charmdev:ubuntu.com)
 
 ## Project and community
-* [Issues](https://github.com/canonical/github-runner-image-builder-operator/issues) <!--Link to GitHub issues (if applicable)-->
-* [Contributing](https://github.com/canonical/github-runner-image-builder-operator/blob/main/CONTRIBUTING.md) <!--Link to any contribution guides--> 
-* [Matrix](https://matrix.to/#/#charmhub-charmdev:ubuntu.com) <!--Link to contact info (if applicable), e.g. Matrix channel-->
+
+- [Issues](https://github.com/canonical/github-runner-image-builder-operator/issues) <!--Link to GitHub issues (if applicable)-->
+- [Contributing](https://github.com/canonical/github-runner-image-builder-operator/blob/main/CONTRIBUTING.md) <!--Link to any contribution guides-->
+- [Matrix](https://matrix.to/#/#charmhub-charmdev:ubuntu.com) <!--Link to contact info (if applicable), e.g. Matrix channel-->

app/src/github_runner_image_builder/cli.py

@@ -90,8 +90,15 @@ def initialize(ctx: click.Context, arch: config.Arch, prefix: str) -> None:
 
 @main.command(name="latest-build-id")
 @click.argument("image_name")
+@click.option(
+    "--any-status",
+    "any_status",
+    is_flag=True,
+    default=False,
+    help="Return the latest image in any upload status (including saving/queued).",
+)
 @click.pass_context
-def get_latest_build_id(ctx: click.Context, image_name: str) -> None:
+def get_latest_build_id(ctx: click.Context, image_name: str, any_status: bool) -> None:
     # Click arguments do not take help parameter, display help through docstrings.
     """Get latest build ID of <IMAGE_NAME> from Openstack <--os-cloud>.
 
@@ -100,10 +107,13 @@ def get_latest_build_id(ctx: click.Context, image_name: str) -> None:
     Args:
         ctx: click.Context object for passing shared state.
         image_name: The image name uploaded to Openstack.
+        any_status: If True, return the latest image in any upload status.
     """  # noqa: D301 - the \f should not be escaped for click to properly format the docstring.
     state = cast(SharedState, ctx.obj)
     click.echo(
-        message=store.get_latest_build_id(cloud_name=state.cloud, image_name=image_name),
+        message=store.get_latest_build_id(
+            cloud_name=state.cloud, image_name=image_name, active_only=not any_status
+        ),
         nl=False,
     )
 

app/src/github_runner_image_builder/store.py

@@ -123,32 +123,40 @@ def _prune_old_images(
             raise OpenstackError from exc
 
 
-def get_latest_build_id(cloud_name: str, image_name: str) -> str:
+def get_latest_build_id(cloud_name: str, image_name: str, active_only: bool = True) -> str:
     """Fetch the latest image id.
 
     Args:
         cloud_name: The Openstack cloud to use from clouds.yaml.
         image_name: The image name to search for.
+        active_only: If True (default), only return active images. If False, return the
+            latest image in any upload status (including saving/queued).
 
     Returns:
-        The image ID if exists, None otherwise.
+        The image ID if exists, empty string otherwise.
     """
     with openstack.connect(cloud=cloud_name) as connection:
-        images = _get_sorted_images_by_created_at(connection=connection, image_name=image_name)
+        images = _get_sorted_images_by_created_at(
+            connection=connection, image_name=image_name, active_only=active_only
+        )
         if not images:
             return ""
         # The type of ID is in string but the library does not provide correct type hints for it.
         return images[0].id  # type: ignore
 
 
 def _get_sorted_images_by_created_at(
-    connection: openstack.connection.Connection, image_name: str
+    connection: openstack.connection.Connection,
+    image_name: str,
+    active_only: bool = True,
 ) -> list[Image]:
     """Fetch the images sorted by created_at date.
 
     Args:
         connection: The connected openstack cloud instance.
         image_name: The image name to search for.
+        active_only: If True (default), query only active images via search_images.
+            If False, query all images regardless of status via the image proxy API.
 
     Raises:
         OpenstackError: if there was an error fetching the images.
@@ -157,7 +165,10 @@ def _get_sorted_images_by_created_at(
         The images sorted by created_at date with latest first.
     """
     try:
-        images = cast(list[Image], connection.search_images(image_name))
+        if active_only:
+            images = cast(list[Image], connection.search_images(image_name))
+        else:
+            images = list(connection.image.images(name=image_name))
     except openstack.exceptions.OpenStackCloudException as exc:
         logger.exception("Failed to search images with name %s.", image_name)
         raise OpenstackError from exc

app/tests/unit/test_cli.py

@@ -138,20 +138,34 @@ def test_invalid_latest_build_id_args(cli_runner: CliRunner):
     assert "Error: Missing argument " in result.output
 
 
-def test_latest_build_id(monkeypatch: pytest.MonkeyPatch, cli_runner: CliRunner):
+@pytest.mark.parametrize(
+    "extra_args, expected_active_only",
+    [
+        pytest.param([], True, id="default active-only"),
+        pytest.param(["--any-status"], False, id="any-status flag"),
+    ],
+)
+def test_latest_build_id(
+    monkeypatch: pytest.MonkeyPatch,
+    cli_runner: CliRunner,
+    extra_args: list[str],
+    expected_active_only: bool,
+):
     """
-    arrange: given valid latest-build-id args.
+    arrange: given valid latest-build-id args (with and without --any-status).
     act: when cli is invoked with latest-build-id.
-    assert: latest-build-id is returned.
+    assert: store.get_latest_build_id is called with the correct active_only value.
     """
-    monkeypatch.setattr(
-        cli.store, "get_latest_build_id", MagicMock(return_value=(test_id := "test-id"))
-    )
+    get_latest_mock = MagicMock(return_value=(test_id := "test-id"))
+    monkeypatch.setattr(cli.store, "get_latest_build_id", get_latest_mock)
 
     result = cli_runner.invoke(
-        main, args=[*REQUIRED_MAIN_INPUTS, "latest-build-id", "test-image-name"]
+        main, args=[*REQUIRED_MAIN_INPUTS, "latest-build-id", *extra_args, "test-image-name"]
     )
 
+    get_latest_mock.assert_called_once_with(
+        cloud_name=TEST_CLOUD_NAME, image_name="test-image-name", active_only=expected_active_only
+    )
     assert result.output == test_id
 
 

app/tests/unit/test_store.py

@@ -98,6 +98,47 @@ def test__get_sorted_images_by_created_at(mock_connection: MagicMock):
     ) == [third, second, first]
 
 
+def test__get_sorted_images_by_created_at_any_status(mock_connection: MagicMock):
+    """
+    arrange: given a mocked openstack connection returning images via image proxy.
+    act: when _get_sorted_images_by_created_at is called with active_only=False.
+    assert: connection.image.images is used (not search_images) and result is sorted.
+    """
+    mock_connection.image = MagicMock()
+    mock_connection.image.images.return_value = iter(
+        [
+            (first := MockOpenstackImageFactory(id="1", created_at="2024-01-01T00:00:00Z")),
+            (third := MockOpenstackImageFactory(id="3", created_at="2024-03-03T00:00:00Z")),
+            (second := MockOpenstackImageFactory(id="2", created_at="2024-02-02T00:00:00Z")),
+        ]
+    )
+
+    result = store._get_sorted_images_by_created_at(
+        connection=mock_connection, image_name="test-image", active_only=False
+    )
+
+    mock_connection.image.images.assert_called_once_with(name="test-image")
+    mock_connection.search_images.assert_not_called()
+    assert result == [third, second, first]
+
+
+def test__get_sorted_images_by_created_at_any_status_error(mock_connection: MagicMock):
+    """
+    arrange: given a mocked openstack connection that raises on image proxy call.
+    act: when _get_sorted_images_by_created_at is called with active_only=False.
+    assert: OpenstackError is raised.
+    """
+    mock_connection.image = MagicMock()
+    mock_connection.image.images.side_effect = openstack.exceptions.OpenStackCloudException(
+        "Network error"
+    )
+
+    with pytest.raises(OpenstackError):
+        store._get_sorted_images_by_created_at(
+            connection=mock_connection, image_name=MagicMock, active_only=False
+        )
+
+
 def test__prune_old_images_error(mock_connection: MagicMock):
     """
     arrange: given a mocked delete function that raises an exception.

charmcraft.yaml

@@ -47,10 +47,9 @@ parts:
       tar -cvzf app.tar.gz app
       cp app.tar.gz /root/stage
     organize:
-        app.tar.gz: app.tar.gz
+      app.tar.gz: app.tar.gz
     prime:
-        - app.tar.gz
-
+      - app.tar.gz
 
 bases:
   - build-on:
@@ -107,22 +106,13 @@ config:
         The auth_url section of the clouds.yaml contents, used to authenticate the OpenStack \
         client (e.g. http://my-openstack-deployment/openstack-keystone). See https://docs.\
         openstack.org/python-openstackclient/queens/configuration/index.html for more information.
-    openstack-password:
-      type: string
-      default: ""
-      description: |
-        The password section of the clouds.yaml contents, used to authenticate the OpenStack \
-        client (e.g. myverysecurepassword). See https://docs.openstack.org/python-openstackclient/\
-        queens/configuration/index.html for more information.
-        DEPRECATED: Use openstack-password-secret instead for better security.
     openstack-password-secret:
       type: secret
       description: |
         The password section of the clouds.yaml contents, used to authenticate the OpenStack
         client. A Juju user secret ID should be passed in the format of secret:<secret-id>.
         The secret must contain a 'password' key with the OpenStack password as its value.
         Example: juju add-secret openstack-password password=<value>.
-        This option takes precedence over openstack-password if both are set.
     openstack-project-domain-name:
       type: string
       default: ""