Implement integration test for airflowctl with 3.1 #56124

bugraoz93 · 2025-09-26T00:12:33Z

closes: #54243
Runs API Server
Executes airflowctl commands listed
It controls generic error, aligns with airflowctl client error handling
If no error means the command was successful

Future work, as an enhancement, we can add expected response as well, but I think this should already cover integration with the API without getting an error from the API.

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

bugraoz93 · 2025-09-27T10:44:35Z

Running locally but until today morning, I have been trying to integrate the keyring into the prod image. I will make a debug/test mode in airflowctl to disable keyring integration for debugging. It is not to include this integration test. This can purely be with Airflow and API. I think we can add additional keyring integration test later on which is another tool and interaction with only 2 lines one for set and one for get, that could be done in a more isolated environment and we don't fill these dependencies in our prod image like gnome-keyring and all the dbus libs which are not related. This makes me think we should provide lighter images for airflowctl to enable easier automations and test these kinds of things with that image

bugraoz93 · 2025-09-27T13:31:23Z

I undo those commits and enabled debug mode while warning users not to use it if they are not debugging and a warning is raised before each command run they are in debug mode. Maybe we can have airflowctl debug clean or something clean the created files for token

gopidesupavan · 2025-10-17T20:48:05Z

will take a look today :)

dev/breeze/doc/05_test_commands.rst

gopidesupavan

LGTM overall

airflow-ctl-tests/docker/docker-compose.yaml

airflow-ctl-tests/pyproject.toml

potiuk

NIce - few small things to address.

bugraoz93 · 2025-10-18T17:24:55Z

Thanks a lot for the comments! I will address them soon :)

airflow-ctl-tests/pyproject.toml

potiuk

Nice!!!!

bugraoz93 · 2025-10-19T21:16:24Z

Strange, it failed to import python on whales, maybe due to the pytest context. I will try to move Docker creation to conftest in an isolated method

…gration tests to skip keyring * Warn user while running each command if debug mode enabled and explicitly state it shouldn't be used unless debugging or integration tests

…e, update documentation mistakes

Co-authored-by: Jarek Potiuk <jarek@potiuk.com>

… in conftest

bugraoz93 · 2025-10-19T22:28:41Z

All of them install devel-common on pytest init but not airflowctl. Their only difference is having the airflow-core. It could come from there, will check what the difference is

bugraoz93 · 2025-10-19T22:42:35Z

Reproduced locally, seems airflow-ctl-tests/ not using the same venv and trying to create it's own

Good version of Docker: 28.3.3.
Good version of docker-compose: 2.39.1
Executable permissions on entrypoints are OK
Running airflowctl integration tests with PROD image: ghcr.io/apache/airflow/main/prod/python3.10
Using airflowctl version: 1.0.0
warning: `VIRTUAL_ENV=/home/bugra/PycharmProjects/airflow/.venv` does not match the project environment path `.venv` and will be ignored; use `--active` to target the active environment instead
ImportError while loading conftest '/home/bugra/PycharmProjects/airflow/airflow-ctl-tests/tests/airflowctl_tests/conftest.py'.
tests/airflowctl_tests/conftest.py:24: in <module>
    from python_on_whales import DockerClient, docker
E   ModuleNotFoundError: No module named 'python_on_whales'

bugraoz93 · 2025-10-19T23:14:41Z

Okay, it seems an uv workspace issue, I need to update the main pyproject.toml. I will push fix soon

…nsole to singleton __init__.py

potiuk · 2025-10-20T08:44:01Z

Okay, it seems an uv workspace issue, I need to update the main pyproject.toml. I will push fix soon

Nice!

github-actions · 2025-10-20T08:45:12Z

Backport failed to create: v3-1-test. View the failure log Run details

Status	Branch	Result
❌	v3-1-test

You can attempt to backport this manually by running:

cherry_picker 80911e8 v3-1-test

This should apply the commit to the v3-1-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

* Implement initial integration test for airflowctl with 3.1 * password can be passed without interaction, update integration tests * Add AIRFLOW_CLI_DEBUG_MODE for enhanced CLI debugging and update integration tests to skip keyring * Warn user while running each command if debug mode enabled and explicitly state it shouldn't be used unless debugging or integration tests * Move python-on-whales to devel-common, use shared docker-composer file, update documentation mistakes * remove shared python-on-whales from airflow-ctl-tests/ Co-authored-by: Jarek Potiuk <jarek@potiuk.com> * Decouple docker compose logic from test method to pytest_sessionstart in conftest * Move python_on_whale import to file level * Reorder dependencies in pyproject.toml for consistency * Add workspace to main pyproject.toml, remove unused variable, move console to singleton __init__.py * Add workspace to main pyproject.toml, remove unused variable, move console to singleton __init__.py --------- Co-authored-by: Jarek Potiuk <jarek@potiuk.com>

* Implement initial integration test for airflowctl with 3.1 * password can be passed without interaction, update integration tests * Add AIRFLOW_CLI_DEBUG_MODE for enhanced CLI debugging and update integration tests to skip keyring * Warn user while running each command if debug mode enabled and explicitly state it shouldn't be used unless debugging or integration tests * Move python-on-whales to devel-common, use shared docker-composer file, update documentation mistakes * remove shared python-on-whales from airflow-ctl-tests/ Co-authored-by: Jarek Potiuk <jarek@potiuk.com> * Decouple docker compose logic from test method to pytest_sessionstart in conftest * Move python_on_whale import to file level * Reorder dependencies in pyproject.toml for consistency * Add workspace to main pyproject.toml, remove unused variable, move console to singleton __init__.py * Add workspace to main pyproject.toml, remove unused variable, move console to singleton __init__.py --------- (cherry picked from commit 80911e8) Co-authored-by: Bugra Ozturk <bugraoz93@users.noreply.github.com> Co-authored-by: Jarek Potiuk <jarek@potiuk.com>

* Implement initial integration test for airflowctl with 3.1 * password can be passed without interaction, update integration tests * Add AIRFLOW_CLI_DEBUG_MODE for enhanced CLI debugging and update integration tests to skip keyring * Warn user while running each command if debug mode enabled and explicitly state it shouldn't be used unless debugging or integration tests * Move python-on-whales to devel-common, use shared docker-composer file, update documentation mistakes * remove shared python-on-whales from airflow-ctl-tests/ Co-authored-by: Jarek Potiuk <jarek@potiuk.com> * Decouple docker compose logic from test method to pytest_sessionstart in conftest * Move python_on_whale import to file level * Reorder dependencies in pyproject.toml for consistency * Add workspace to main pyproject.toml, remove unused variable, move console to singleton __init__.py * Add workspace to main pyproject.toml, remove unused variable, move console to singleton __init__.py --------- Co-authored-by: Jarek Potiuk <jarek@potiuk.com>

* Implement initial integration test for airflowctl with 3.1 * password can be passed without interaction, update integration tests * Add AIRFLOW_CLI_DEBUG_MODE for enhanced CLI debugging and update integration tests to skip keyring * Warn user while running each command if debug mode enabled and explicitly state it shouldn't be used unless debugging or integration tests * Move python-on-whales to devel-common, use shared docker-composer file, update documentation mistakes * remove shared python-on-whales from airflow-ctl-tests/ Co-authored-by: Jarek Potiuk <jarek@potiuk.com> * Decouple docker compose logic from test method to pytest_sessionstart in conftest * Move python_on_whale import to file level * Reorder dependencies in pyproject.toml for consistency * Add workspace to main pyproject.toml, remove unused variable, move console to singleton __init__.py * Add workspace to main pyproject.toml, remove unused variable, move console to singleton __init__.py --------- (cherry picked from commit 80911e8) Co-authored-by: Bugra Ozturk <bugraoz93@users.noreply.github.com> Co-authored-by: Jarek Potiuk <jarek@potiuk.com>

… (#56986) * [v3-1-test] Implement integration test for airflowctl with 3.1 (#56124) * Implement initial integration test for airflowctl with 3.1 * password can be passed without interaction, update integration tests * Add AIRFLOW_CLI_DEBUG_MODE for enhanced CLI debugging and update integration tests to skip keyring * Warn user while running each command if debug mode enabled and explicitly state it shouldn't be used unless debugging or integration tests * Move python-on-whales to devel-common, use shared docker-composer file, update documentation mistakes * remove shared python-on-whales from airflow-ctl-tests/ Co-authored-by: Jarek Potiuk <jarek@potiuk.com> * Decouple docker compose logic from test method to pytest_sessionstart in conftest * Move python_on_whale import to file level * Reorder dependencies in pyproject.toml for consistency * Add workspace to main pyproject.toml, remove unused variable, move console to singleton __init__.py * Add workspace to main pyproject.toml, remove unused variable, move console to singleton __init__.py --------- (cherry picked from commit 80911e8) Co-authored-by: Bugra Ozturk <bugraoz93@users.noreply.github.com> Co-authored-by: Jarek Potiuk <jarek@potiuk.com> * Remove e2e tests came from merge and rerun breeze images for all commands to fix accepting changes while merging * Add env file to compose and update error --------- Co-authored-by: Jarek Potiuk <jarek@potiuk.com>

boring-cyborg bot added area:dev-tools area:production-image Production image improvements and fixes backport-to-v3-1-test Mark PR with this label to backport to v3-1-test branch labels Sep 26, 2025

bugraoz93 force-pushed the airflowctl/integration-tests branch from f0cc805 to 0dc8757 Compare September 26, 2025 21:14

bugraoz93 force-pushed the airflowctl/integration-tests branch 2 times, most recently from ba4c93b to 877d465 Compare September 27, 2025 11:57

bugraoz93 marked this pull request as ready for review September 27, 2025 13:31

bugraoz93 requested review from amoghrajesh, ashb, gopidesupavan, jedcunningham and potiuk as code owners September 27, 2025 13:31

bugraoz93 requested a review from jscheffl September 27, 2025 13:31

bugraoz93 force-pushed the airflowctl/integration-tests branch 2 times, most recently from 4e98c61 to e0be173 Compare October 17, 2025 16:46

bugraoz93 changed the title ~~Implement initial integration test for airflowctl with 3.1~~ Implement integration test for airflowctl with 3.1 Oct 17, 2025

bugraoz93 force-pushed the airflowctl/integration-tests branch from e0be173 to 3e8ed79 Compare October 17, 2025 20:41

gopidesupavan reviewed Oct 18, 2025

View reviewed changes

dev/breeze/doc/05_test_commands.rst Outdated Show resolved Hide resolved

gopidesupavan reviewed Oct 18, 2025

View reviewed changes

dev/breeze/doc/05_test_commands.rst Outdated Show resolved Hide resolved

gopidesupavan approved these changes Oct 18, 2025

View reviewed changes

potiuk reviewed Oct 18, 2025

View reviewed changes

airflow-ctl-tests/docker/docker-compose.yaml Outdated Show resolved Hide resolved

potiuk reviewed Oct 18, 2025

View reviewed changes

airflow-ctl-tests/docker/docker-compose.yaml Outdated Show resolved Hide resolved

potiuk reviewed Oct 18, 2025

View reviewed changes

airflow-ctl-tests/pyproject.toml Outdated Show resolved Hide resolved

potiuk approved these changes Oct 18, 2025

View reviewed changes

bugraoz93 force-pushed the airflowctl/integration-tests branch from 3e8ed79 to da3ece1 Compare October 19, 2025 17:00

bugraoz93 requested a review from kaxil as a code owner October 19, 2025 17:00

bugraoz93 requested review from eladkal, ephraimbuddy and jason810496 as code owners October 19, 2025 17:00

potiuk reviewed Oct 19, 2025

View reviewed changes

airflow-ctl-tests/pyproject.toml Outdated Show resolved Hide resolved

potiuk approved these changes Oct 19, 2025

View reviewed changes

bugraoz93 force-pushed the airflowctl/integration-tests branch from c71bfec to 9bd9e3f Compare October 19, 2025 17:48

bugraoz93 and others added 6 commits October 19, 2025 23:38

Implement initial integration test for airflowctl with 3.1

950332b

password can be passed without interaction, update integration tests

8a818c0

Add AIRFLOW_CLI_DEBUG_MODE for enhanced CLI debugging and update inte…

34c1817

…gration tests to skip keyring * Warn user while running each command if debug mode enabled and explicitly state it shouldn't be used unless debugging or integration tests

Move python-on-whales to devel-common, use shared docker-composer fil…

c020771

…e, update documentation mistakes

remove shared python-on-whales from airflow-ctl-tests/

6722d82

Co-authored-by: Jarek Potiuk <jarek@potiuk.com>

Decouple docker compose logic from test method to pytest_sessionstart…

c6d8436

… in conftest

bugraoz93 force-pushed the airflowctl/integration-tests branch from 9bd9e3f to c6d8436 Compare October 19, 2025 21:38

Move python_on_whale import to file level

aec0823

Reorder dependencies in pyproject.toml for consistency

8ec32f8

bugraoz93 added 2 commits October 20, 2025 01:20

Add workspace to main pyproject.toml, remove unused variable, move co…

c6fd575

…nsole to singleton __init__.py

Add workspace to main pyproject.toml, remove unused variable, move co…

d3e3e2e

…nsole to singleton __init__.py

potiuk merged commit 80911e8 into apache:main Oct 20, 2025
207 checks passed

potiuk mentioned this pull request Nov 8, 2025

Add missing translations for PL for 3.1.3 #58076

Merged

Implement integration test for airflowctl with 3.1 #56124

Implement integration test for airflowctl with 3.1 #56124

Uh oh!

Conversation

bugraoz93 commented Sep 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bugraoz93 commented Sep 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bugraoz93 commented Sep 27, 2025

Uh oh!

gopidesupavan commented Oct 17, 2025

Uh oh!

Uh oh!

Uh oh!

gopidesupavan left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

potiuk left a comment

Choose a reason for hiding this comment

Uh oh!

bugraoz93 commented Oct 18, 2025

Uh oh!

Uh oh!

potiuk left a comment

Choose a reason for hiding this comment

Uh oh!

bugraoz93 commented Oct 19, 2025

Uh oh!

bugraoz93 commented Oct 19, 2025

Uh oh!

bugraoz93 commented Oct 19, 2025

Uh oh!

bugraoz93 commented Oct 19, 2025

Uh oh!

potiuk commented Oct 20, 2025

Uh oh!

Uh oh!

github-actions bot commented Oct 20, 2025

Backport failed to create: v3-1-test. View the failure log Run details

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

bugraoz93 commented Sep 26, 2025 •

edited

Loading

bugraoz93 commented Sep 27, 2025 •

edited

Loading