-
Notifications
You must be signed in to change notification settings - Fork 16.4k
Add local airflow + configuration cleanup #37644
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Our CI sometimes reuses same runner for several jobs (this is until we switch to --ephemeral runners) and this might lead to some side-effects between runners. When the two runners are run for different branches, and airflow is locally installed, it might lead to configuration being written by one version and used by another. This - for example - made https://github.com/apache/airflow/actions/runs/8011411765/job/21884630103 to fail. This PR adds extra cleanup to `breeze ci free-space` command and add extra step in `breeze cleanup` to remove installed airflow and the whole airflow home.
|
I used breeze cleanup just now and realized that it deletes everything under the airflow path. Since this also includes the repo .git folder this could be risky for developers if they expect it to clean up docker and build related stuff. |
|
It's a very bad idea to checkout airflow in |
|
Or so I thought: PR adding protection against such checkout is here: #37697 After it's merged, when you checkout your Airlfow in HOME/airflow and use breeze you will see this (and you won't be able to run breeze)
:
|
|
Ok thanks I wasn't aware of this |
So now it will be clear to anyone who is not aware :) |

Our CI sometimes reuses same runner for several jobs (this is until we switch to --ephemeral runners) and this might lead to some side-effects between runners. When the two runners are run for different branches, and airflow is locally installed, it might lead to configuration being written by one version and used by another. This - for example - made
https://github.com/apache/airflow/actions/runs/8011411765/job/21884630103 to fail.
This PR adds extra cleanup to
breeze ci free-spacecommand and add extra step inbreeze cleanupto remove installed airflow and the whole airflow home.^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in newsfragments.