Skip to content

ci: require secrets.PYCLOUDLIB_TOML, add lxd_vm and ec2 scheduled jobs#6715

Open
blackboxsw wants to merge 7 commits into
canonical:mainfrom
blackboxsw:ci-lxd-vm
Open

ci: require secrets.PYCLOUDLIB_TOML, add lxd_vm and ec2 scheduled jobs#6715
blackboxsw wants to merge 7 commits into
canonical:mainfrom
blackboxsw:ci-lxd-vm

Conversation

@blackboxsw
Copy link
Copy Markdown
Collaborator

@blackboxsw blackboxsw commented Feb 5, 2026

Improve our public scheduled integration test coverage to cover lxd_vm and ec2 platforms.

Running integration tests will require 3 repository secrets PYCLOUDLIB_TOML_B64, SSH_PUBLIC_KEY and SSH_PRIVATE_KEY in order to configure pycloudlib to run integration tests.

This PR renames and orders Github workflow files to aid visibility when manually reviewing and running actions.

Add assertion on a non-empty Github repo-level secret named PYCLOUDLIB_TOML_B64 which will
be base64 decoded and written to a temporary file which is set in the environment variable PYCLOUDLIB_CONFIG. This config is used sa runtime integration test configuration for pycloudlib.

Add an assertion step in 20-dispatch-common.yml to error when
secrets.PYCLOUDLIB_TOML_B64 is absent and conditional logic which skips
each integration test step to avoid wasting runner cycles on integration tests.

Proposed Commit Message

See individual commits

Additional Context

Test Steps

Failed run example empty or absent secret PYCLOUDLIB_TOML_B64 https://github.com/blackboxsw/cloud-init/actions/runs/21699149948/job/62575866822

Success RUN with PYCLOUDLIB_TOML_B64 secret set to "[ec2]"
https://github.com/blackboxsw/cloud-init/actions/runs/24084983529/job/70255277685

Merge type

  • Squash merge using "Proposed Commit Message"
  • Rebase and merge unique commits. Requires commit messages per-commit each referencing the pull request number (#<PR_NUM>)

@blackboxsw blackboxsw force-pushed the ci-lxd-vm branch 2 times, most recently from 4f73b0f to ed8690d Compare February 5, 2026 05:20
@blackboxsw blackboxsw marked this pull request as draft February 5, 2026 05:21
Copy link
Copy Markdown
Member

@holmanb holmanb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

first pass

Comment on lines +94 to +96
awk '/cloud-init version: /{printf DEB_VERSION=$NF; exit}' pytest-${{ inputs.platform }}-${{ inputs.release }}-${{ inputs.image_type }}.log
awk '/image-serial: /{printf IMAGE_SERIAL=$NF; exit}' pytest-${{ inputs.platform }}-${{ inputs.release }}-${{ inputs.image_type }}.log
shell: bash
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is are the awk commands doing here?

And why bash?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahh I was going to use these to help create a top-level report output that would announce what version of cloud-init was installed during the test. I'll drop this from this PR until I have a working approach. I didn't like how opaque our GH workflow runs are when compared to jenkins jobs which announce the version of cloud-init being tested.

Copy link
Copy Markdown
Collaborator Author

@blackboxsw blackboxsw Feb 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was originally stuffing env vars them into GITHUB_ENV per these docs and the ctrf.io step can extract and use the environment variables for report headers or summary.

@holmanb holmanb self-assigned this Feb 5, 2026
@holmanb holmanb added the incomplete Action required by submitter label Feb 10, 2026
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Setup LXD
if: ${{ env.REQUIRED_SECRET != '' and contains(fromJSON('["lxd_vm", "lxd_container"]'), env.CLOUD_INIT_PLATFORM ) }}
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Avoid the setup-lxd step if we are not on CLOUD_INIT_OS_PLATFORM lxd_container of lxd_vm.

@blackboxsw blackboxsw requested a review from holmanb February 11, 2026 23:27
@blackboxsw blackboxsw removed the incomplete Action required by submitter label Feb 11, 2026
@blackboxsw blackboxsw marked this pull request as ready for review February 13, 2026 04:32
Comment on lines +67 to +68
test '${{ secrets.PYCLOUDLIB_TOML }}' != '' || echo "ERROR: Missing required repo secrets.PYCLOUDLIB_TOML non-empty value."
test '${{ secrets.PYCLOUDLIB_TOML }}' == '' && exit 1
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This checks for the same thing twice, I think an if / else would be cleaner

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. We only really care to exit 1 in the face of no secrets.PYCLOUDLIB_TOML or empty string. So, I just put the operation in a single if clause.

- name: Checkout
if: ${{ env.REQUIRED_SECRET != '' }}
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since env.REQUIRED_SECRET is assigned from secrets.PYCLOUDLIB_TOML, and the first step has an exit 1 when secrets.PYCLOUDLIB_TOML is empty, is this not redundant?

Same comment elsewhere.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dropped all unneessary conditionals checks. An exit 1 above will prevent running these steps anyway.

@holmanb holmanb added the incomplete Action required by submitter label Feb 26, 2026
@blackboxsw blackboxsw force-pushed the ci-lxd-vm branch 7 times, most recently from 0c75a04 to 3937aae Compare March 10, 2026 21:15
@holmanb
Copy link
Copy Markdown
Member

holmanb commented Mar 16, 2026

@blackboxsw let me know when this is ready for re-review

@github-actions github-actions Bot added the stale-pr Pull request is stale; will be auto-closed soon label Mar 31, 2026
@canonical canonical deleted a comment from github-actions Bot Apr 6, 2026
Copy link
Copy Markdown
Member

@holmanb holmanb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few comments.

Comment thread .github/workflows/08-daily-integration-26.04-ec2.yml Outdated
@@ -0,0 +1,13 @@
name: "Daily: Integration Resolute lxd_vm"
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The naming scheme isn't going to work with future expansion of new series, additional platforms, etc.

I would prefer that we do it in a way that isn't going to force us to do the same thing again the next time we add more platforms - something a little bit more sustainable would be better.

Also - numbers simultaneously increasing and decreasing is chaotic.

Copy link
Copy Markdown
Collaborator Author

@blackboxsw blackboxsw Apr 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have dropped all unrelated renames and moved all integration platform runs to the end of the ordered list. Each platform is now grouped and reserves 10 workflow test slots for each platform.
Platform group prefix will be the following for each platform:

  • 100: lxd_container
  • 110: lxd_vm
  • 120: ec2

The oldest tested series (22.04) will be the first index at prefix 100-. Each additional supported series will increment from that initial platform index:

  • XX0: 22.04
  • XX1: 24.04
  • XX2: 25.10
  • XX3: 26.04

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@github-actions github-actions Bot removed the stale-pr Pull request is stale; will be auto-closed soon label Apr 7, 2026
@blackboxsw blackboxsw requested a review from holmanb April 7, 2026 14:11
@blackboxsw blackboxsw removed the incomplete Action required by submitter label Apr 7, 2026
@blackboxsw
Copy link
Copy Markdown
Collaborator Author

@holmanb I have updated this branch and linked a successful runs on Ec2.

Rename the reusable dispatch workflow to 100-dispatch-common.yml to
establish a convention where 1xx = reusable/callable workflows.

Workflow indexes 11x/12x/13x are caller workflows grouped by platform:
  lxd_container, lxd_vm and ec2.

Move scheduled daily-integration-*-lxd-container.yml files to
11x-daily-integration-*-lxd_container.yml equivalent.

Each platform group of scheduled workflows will be ordered by
increasing Ubuntu series: 22.04, 24.04, 25.10, 26.04.
…er temp

- Declare PYCLOUDLIB_TOML, SSH_PRIVATE_KEY, and SSH_PUBLIC_KEY as
  required secrets on the workflow_call trigger.
- Add an early assertion step that exits immediately if
  PYCLOUDLIB_TOML is empty.
- Replace the ssh-keygen throwaway key with a new 'Setup SSH' step
  that writes the injected public/private key pair to ~/.ssh, for use
  in authentication to instances under test
- base64-decode PYCLOUDLIB_TOML secret and write it to
  $RUNNER_TEMP/pycloudlib.toml to keep secrets off the persistent filesystem.
  Add cleanup to remove the file unconditionally on exit.
Add workflow files 120-123 for lxd_vm, daily, on all supported releases.
Add workflow files 130-133 for EC2, twice weekly, all supported releases

All new caller workflows expose an install_source workflow_dispatch
input, defaulting to ppa:cloud-init-dev/daily, and forward the
three required secrets to the common dispatch workflow.
Avoid dependency on ctrf.io integrations and shift to pytest-json-ctrf
for report formatting needs.

- Add pytest-json-ctrf to integration-requirements.txt; this plugin
  writes a native CTRF JSON report file and provide --ctrf to pytest.
- Set a descriptive title that includes  platform, release, image_type,
  and install_source.

Job and environment cleanup:
- Move CLOUD_INIT_OS_IMAGE, CLOUD_INIT_OS_IMAGE_TYPE,
  CLOUD_INIT_CLOUD_INIT_SOURCE, and CLOUD_INIT_LOCAL_LOG_PATH from
  the job-level env block into the 'Run integration Tests' step env,
  keeping the job env focused on credentials and config that steps
  other than the test runner need.
- Add a default of 'ppa:cloud-init-dev/daily' to the workflow_call
  install_source input so callers that omit it still get a sensible
  value.
- Add install_source workflow_dispatch input with the same default to
  the lxd_container caller workflows (110-113) and pass it through.
@blackboxsw
Copy link
Copy Markdown
Collaborator Author

@holmanb I believe I have addressed all questions or suggestions. I have refactored the commits into logically separate commits for ease of review.

Final successful run https://github.com/blackboxsw/cloud-init/actions/runs/24911475438/job/72953869594.

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 19 out of 19 changed files in this pull request and generated 5 comments.

uses: ./.github/workflows/100-dispatch-common.yml
with:
release: resolute
platform: ec2
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see now that an issue in one of my latest comments was introduced by following one of the recommendations suggested here.

Please self-review for consistency with the existing code when applying suggested changes.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given that we now decided below to shift this to a required parameter, all workflows will provide image_type for both cheduled and manual workflow dispatch.

Comment thread .github/workflows/100-dispatch-common.yml Outdated
Comment on lines +103 to +104
run: |
sh -c 'echo "${{ secrets.PYCLOUDLIB_TOML}}" | base64 -d > "$PYCLOUDLIB_CONFIG"'
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll prefer to validate that the content is base64-encoded. Will perform this validation by checking for a non-empty decoded PYCLOUDLIB_CONFIG after processing.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After review, we'll allow the base64 -d failure and non-zero exit code leave us with an adequate breadcrumb to resolve this.

options:
- generic
- minimal
required: false
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would have suggested making this required instead

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shifted to required: true. All scheduled workflows provide image_type: generic currently.

uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Setup LXD
if: ${{ contains(fromJSON('["lxd_vm", "lxd_container"]'), env.CLOUD_INIT_PLATFORM ) }}
uses: canonical/setup-lxd@8c6a87bfb56aa48f3fb9b830baa18562d8bfd4ee # v0.1.2
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it has a point.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it has a point.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm uncertain what point you think has been missed here. I have updated the commit hash and comment to align with the upstream repo tag # v1. This is already resolved. in the second commit from the top. 9cd0ff6.

- name: Checkout
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Setup LXD
if: ${{ contains(fromJSON('["lxd_vm", "lxd_container"]'), env.CLOUD_INIT_PLATFORM ) }}
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This workflow is shared and generic. Why is an lxd specialization used here?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@holmanb the lxd specialization is here because we only want to install the lxd snaps if we are launching LXD backends for integration testing, Azure,GCE and Ec2 don't need this step. I've added a comment.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand why it is needed. But I still don't understand why it is needed here.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is in 100-dispatch-common because we allow manual "Run workflow" dispatch directly from this workflow providing a dropdown choice to select the platform which can be lxd_vm or lxd_container. As a result, this conditional logic needs to be in here to allow that flexibility for a manual run which can include lxd_*

# Dump secrets using a subprocess to avoid accidental leaks while debugging.
sh -c 'printf "%s\n" "$SSH_PUBLIC_KEY" > ~/.ssh/cloudinit_id_rsa.pub'
sh -c 'printf "%s\n" "$SSH_PRIVATE_KEY" > ~/.ssh/cloudinit_id_rsa'
chmod 600 ~/.ssh/cloudinit_id_rsa
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: where code fails on err, this is probably fine

but as a pattern it is undesirable to chmod after writing your secret, because there is a window of time between the two commands where the secret can be accessed by code with lower permissions

Copy link
Copy Markdown
Collaborator Author

@blackboxsw blackboxsw May 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good thought. Fixed with an early touch ~/.ssh/cloudinit_id_rsa && chmod 600 ~/.ssh/cloudinit_id_rsa

@blackboxsw
Copy link
Copy Markdown
Collaborator Author

blackboxsw commented May 6, 2026

@blackboxsw blackboxsw requested a review from holmanb May 6, 2026 16:18
@blackboxsw
Copy link
Copy Markdown
Collaborator Author

@holmanb all your comments and copilot's are addressed. I've commented iwth success runs for lxd_vm and lxc_container.

@blackboxsw
Copy link
Copy Markdown
Collaborator Author

#6864 just landed so Alternate distros CI would pass if we rebased.

Address review comments from copilot and Brett:
- update setup-lxd pinning
- validate presence of non-empty required SSH_*_KEY secrets
- simplifiy -z secret test conditionals to avoid multi-line issues
- use install -m 600 before creating SSH_PRIVATE_KEY file
- test and exit 1 on non-encoded PYCLOUDLIB_TOML repo secret
- set image_type: generic as defaults in 100-dispatch-common and ec2
  job.
@blackboxsw
Copy link
Copy Markdown
Collaborator Author

Renamed PYCLOUDLIB_TOML -> PYCLOUDLIB_TOML_B64 to indicate intent for value.
Success run lxd_vm

Copy link
Copy Markdown
Member

@holmanb holmanb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the updates.

There are still some inconsistencies and issues to figure out - and I think that some of these would have been caught by an attentive self-review. See my comments.

# Dump secrets using a subprocess to avoid accidental leaks while debugging.
sh -c 'printf "%s\n" "$SSH_PUBLIC_KEY" > ~/.ssh/cloudinit_id_rsa.pub'
# Create empty cloudinit_id_rsa with file mode 600.
touch ~/.ssh/cloudinit_id_rsa && chmod 600 ~/.ssh/cloudinit_id_rsa
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this would be cleaner: install -m 600 <(echo $var) file

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had used that before, but install command isn't available in the workflow and resulted in an error.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well crud, it was a typo 'intall command not found`. Shifting it back.

PYCLOUDLIB_CONFIG: ${{ runner.temp }}/pycloudlib.toml
CLOUD_INIT_LOCAL_LOG_PATH: ${{ github.workspace }}/cloud_init_test_logs
run: |
tox -e integration-tests -- --ctrf=${{ github.workspace }}/report.json --color=yes ${{ inputs.filter_tests || 'tests/integration_tests' }}
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is a default value conditionally set in test logic?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right, it is unnecessary, when absent we won't provide any test paths and cloud-init integration tests will default to tests/integration_tests

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, in testing, the absence of that default/fallback results in dependency errors from unittests this happens because we are already providing posargs which causes tox to drop the default of tests/integration_tests from tox.ini and try to run all tests in all directories. So, unittest dependencies end up 'missing' from integration-test target in this case.

So, we still need a default of fiilter_tests is unset or empty. Given that our workflow dispatch doesn't have a default, the value of filter_tests is an empty string if unspecified. We need the fallback conditional on an empty value to ensure we pass a reasonable posarg to pyttest for the integration-test target. So this logic needs to remain.

run: |
sh -c 'echo "${{ secrets.PYCLOUDLIB_TOML_B64}}" | base64 -d > "$PYCLOUDLIB_CONFIG"' | true
if [ ! -s $PYCLOUDLIB_CONFIG ]; then
echo "PYCLOUDLIB_TOML_B64 repo secret is not a base64-encoded string"
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would just let it exit on error rather than suppressing the error and trying to add a message.

This message assumes a particular failure mode, when in fact multiple failure modes are possible.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, dropped, it will fail in either case. without a specific message you will still see a `base64 error invalid input"

- name: Assert required repo secrets are set
run: |
if [ -z "$REQUIRED_SECRET" ]; then
echo "ERROR: Missing required repo secrets.PYCLOUDLIB_TOML_B64 non-empty value."
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The error message has some issues.

The non-empty value part is confusing. Also, putting a variable name in an error message means this could unknowingly become stale.

Rather than trying to diagnose the root cause of an issue in an error message, why not just state the problem in a clear way?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have adapted the message to:
ERROR: Missing required repo secret. Please provide the necessary repo secret at ${{ github.repository }}/settings/secrets/action.

type: string
default: 'ppa:cloud-init-dev/daily'
schedule:
# Run Mon & Thurs for high-cost test runs
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are the costs really high?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

higher than lxd* on the local worker. We can measure this once we turn on the new workflows.

with:
release: jammy
platform: ec2
install_source: ${{ inputs.install_source || 'ppa:cloud-init-dev/daily' }}
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't this value conditionally set elsewhere?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, this is only conditionally set if we manually use workflow-dispatch via Run workflow button on the specific job. It is not set automatically for the scheduled workflow runner so we need a default in that case.

Comment thread integration-requirements.txt Outdated
pycloudlib>=1!10.0.2,<1!11

pytest-timeout
pytest-json-ctrf # Used for GH ctrf.io test report dashboard
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like this project is no longer supported?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Meh, this README update happened after opening this original PR, having seen recent releases this year prior to that event. Strange, pypi also has an updated release one month ago, despite this project mentioning unsupported nature now. I guess we can go back to the supported junit-to-ctrf integrations-config that is currently in tip of main.

  • junit-to-ctrf is part of the ctrf projecthttps://github.com/ctrf-io/junit-to-ctrf. And it's under active maintenance with active security advisory handling

Comment thread .github/workflows/111-daily-integration-24.04-lxd_container.yml Outdated
- lxd_container
- lxd_vm
- ec2
image_type:
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this needed?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes each parameter is needed to manually dispatch the workflow as seen when clicking the run workflow button on the workflow dispatch page. Without these values being selectable, we can't manually dispatch the workflow as those required inputs would be missing.

Copy link
Copy Markdown
Member

@holmanb holmanb May 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I see that you made it required. What is it needed for?

@holmanb
Copy link
Copy Markdown
Member

holmanb commented May 6, 2026

Also, I just unresolved some copilot reviews with comments.

@blackboxsw blackboxsw force-pushed the ci-lxd-vm branch 2 times, most recently from ef54ca4 to fbbc886 Compare May 7, 2026 04:11
Copy link
Copy Markdown
Collaborator Author

@blackboxsw blackboxsw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you again @holmanb for finding the gaps or inconsistencies. I responded to everything I've seen open. I 'm not sure what else we need to do with setup-lxd though.

- lxd_container
- lxd_vm
- ec2
image_type:
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes each parameter is needed to manually dispatch the workflow as seen when clicking the run workflow button on the workflow dispatch page. Without these values being selectable, we can't manually dispatch the workflow as those required inputs would be missing.

options:
- generic
- minimal
required: false
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shifted to required: true. All scheduled workflows provide image_type: generic currently.

- name: Assert required repo secrets are set
run: |
if [ -z "$REQUIRED_SECRET" ]; then
echo "ERROR: Missing required repo secrets.PYCLOUDLIB_TOML_B64 non-empty value."
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have adapted the message to:
ERROR: Missing required repo secret. Please provide the necessary repo secret at ${{ github.repository }}/settings/secrets/action.

Comment on lines +103 to +104
run: |
sh -c 'echo "${{ secrets.PYCLOUDLIB_TOML}}" | base64 -d > "$PYCLOUDLIB_CONFIG"'
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After review, we'll allow the base64 -d failure and non-zero exit code leave us with an adequate breadcrumb to resolve this.

uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Setup LXD
if: ${{ contains(fromJSON('["lxd_vm", "lxd_container"]'), env.CLOUD_INIT_PLATFORM ) }}
uses: canonical/setup-lxd@8c6a87bfb56aa48f3fb9b830baa18562d8bfd4ee # v0.1.2
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm uncertain what point you think has been missed here. I have updated the commit hash and comment to align with the upstream repo tag # v1. This is already resolved. in the second commit from the top. 9cd0ff6.

with:
release: jammy
platform: ec2
install_source: ${{ inputs.install_source || 'ppa:cloud-init-dev/daily' }}
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, this is only conditionally set if we manually use workflow-dispatch via Run workflow button on the specific job. It is not set automatically for the scheduled workflow runner so we need a default in that case.

uses: ./.github/workflows/100-dispatch-common.yml
with:
release: resolute
platform: ec2
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given that we now decided below to shift this to a required parameter, all workflows will provide image_type for both cheduled and manual workflow dispatch.

- name: Checkout
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Setup LXD
if: ${{ contains(fromJSON('["lxd_vm", "lxd_container"]'), env.CLOUD_INIT_PLATFORM ) }}
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is in 100-dispatch-common because we allow manual "Run workflow" dispatch directly from this workflow providing a dropdown choice to select the platform which can be lxd_vm or lxd_container. As a result, this conditional logic needs to be in here to allow that flexibility for a manual run which can include lxd_*

# Dump secrets using a subprocess to avoid accidental leaks while debugging.
sh -c 'printf "%s\n" "$SSH_PUBLIC_KEY" > ~/.ssh/cloudinit_id_rsa.pub'
# Create empty cloudinit_id_rsa with file mode 600.
touch ~/.ssh/cloudinit_id_rsa && chmod 600 ~/.ssh/cloudinit_id_rsa
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well crud, it was a typo 'intall command not found`. Shifting it back.

Comment thread .github/workflows/111-daily-integration-24.04-lxd_container.yml Outdated
Migrate away from deprecated pytest-json-ctrf reverting to use
ctrf intrgration junit-to-ctrf.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants