Unify all "breeze" tools under breeze sub-commands #23193
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Member
Author
|
@edithturn @Bowrna - seems that with the recent changes where I convert most of the "ci" to use the new "breeze" commands - moving |
potiuk
force-pushed
the
unify-ci-tools
branch
7 times, most recently
from
1ef8ec6 to
53896f1
Compare
Experimentally we've separated some CI-only tools like freespace and find-newer-dependencies as separate scripts (similarly to initialize-virtualenv) but contrary to the initialize, our tools can and should be run in Breeze's virtualenv. However we are just in the middle of using breeze commands in the CI context, the commands are not "alien" any more and they seem to fit the same pattern as other commands. Since we now have the possibility of grouping sub-commmands in separate logical groups, it makes perfect sense to bring the tools to the breeze's sub-commands - mostly because of the auto-complete support and familiarity of using breeze commands in CI. This PR brings both freespace and find-newer-dependencies tools already used in CI to become breeze sub-commands. It also adds "fix-ownership" and "resource-check" commands, so that we can run owership fixing on Linux machines (and adds fixing ownership as mandatory stop in all breeze-run jobs in CI and that users can run resource-check locally. This is because thgere were some cases after introducing Breeze that some root-owned left-overs prevented to checkout code by the runner on self-hosted runners. This was likely due to resource-check that was run without PYTHONDONTWRITEBYTECODE=1 variable - this variable have been added now to execution of the python script.
| * Checking available resources for docker with ``breeze resource-check`` command | ||
| * Freeing space needed to run CI tests with ``breeze free-space`` command | ||
| * Fixing ownership of files in your repository with ``breeze fix-ownership`` command | ||
| * Print Breeze version with ``breeze fix-ownership`` command |
Contributor
There was a problem hiding this comment.
Suggested change
| * Print Breeze version with ``breeze fix-ownership`` command | |
| * Print Breeze version with ``breeze version`` command |
Should it be breeze version?
ephraimbuddy
approved these changes
Apr 24, 2022
|
The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease. |
github-actions
bot
added
the
full tests needed
potiuk
added a commit
to potiuk/airflow
that referenced
this pull request
Apr 24, 2022
Static checks were not really "enabled' after migration since apache#23193 This PR fixes it.
potiuk
added a commit
that referenced
this pull request
Apr 24, 2022
Static checks were not really "enabled' after migration since #23193 This PR fixes it.
jedcunningham
added
the
changelog:skip
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Experimentally we've separated some CI-only tools like freespace
and find-newer-dependencies as separate scripts (similarly to
initialize-virtualenv) but contrary to the initialize, our tools
can and should be run in Breeze's virtualenv.
However we are just in the middle of using breeze commands in
the CI context, the commands are not "alien" any more and
they seem to fit the same pattern as other commands.
Since we now have the possibility of grouping sub-commmands in separate
logical groups, it makes perfect sense to bring the tools to the
breeze's sub-commands - mostly because of the auto-complete
support and familiarity of using breeze commands in CI.
This PR brings both freespace and find-newer-dependencies tools
already used in CI to become breeze sub-commands. It also
adds "fix-ownership" command, so that we can run owership
fixing on Linux machines (and adds fixing ownership as
mandatory stop in all breeze-run jobs in CI.
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragement file, named
{pr_number}.significant.rst, in newsfragments.