Prepare Breeze2 for prime time :) #22713

potiuk · 2022-04-03T17:57:06Z

This is a review and clean-up for all the parameters and
commands for Breeze2 in order to prepare it for being
used by the contribugors.

There are various small fixes here and there, removal
of duplicated code, refactoring and moving code around
as well as cleanup and review all the parameters used
for all implemented commands.

The parameters, default values and their behaviours were
updated to match "new" life of Breeze rather than old
one.

Some improvements are made to the autocomplete and
click help messages printed. Full list of choices is
always displayed, parameters are groups according to
their target audience, and they were sorted according
to importance and frequency of use.

Various messages have been colourised according to their
meaning - warnings as yellow, errors as red and
informational messages as bright_blue.

The dry-run option has been added to just show what
would have been run without actually running some
potentially "write" commands (read commands are still
executed) so that you can easily verify and manually
copy and execute the commands with option to modify
them before. The dry_run and verbose options are
now used for all commands.

The "main" command now runs "shell" by default similarly
as the original Breeze.

All "shortcut" parameters have been standardized - i.e
common options (verbose/dry run/help) have one and all
common flags that are likely to be used often have an
assigned shortcute.

The "stop" and "cleanup" command have been added
as they are necessary for average user to complete the
regular usage cycle.

Documentation for all the important methods have been
updated.

This one is based on #22695 so check only last commit

^ Add meaningful description above

Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.

potiuk · 2022-04-03T18:01:02Z

cc: @Bowrna @edithturn -> I have deeeply reviewed, tested and cleaned up the changes.

This is really cool what we have, after I removed some duplications, cleaned up some naming, added some docstrings and reviewed and made sure all the parameters we have, we are 100% ready to replace old breeze and move the old breeze to legacy. This will be the next PR after the #22695 and this one is merged.

My changes were mostly about beautifying things, refactoring/shuffling them around so that they make more sense and making sure we have no duplications. All the rest remained largely as You've implemented it.

FANTASTIC JOB. Some cool screenshots follow.

potiuk · 2022-04-03T18:06:21Z

Auttocomplete:

Autocomplete 2:

Main help with Breeze2:

Build image help:

Start-airflow help:

Cheatsheet:

eladkal

🎉🎉🎉🎉

eladkal · 2022-04-03T18:34:17Z

CONTRIBUTING.rst


 In Breeze - which is a development environment, ``INSTALL_PROVIDERS_FROM_SOURCES`` variable is set to true,
-but you can add ``--skip-installing-airflow-providers-from-sources`` flag to Breeze to skip installing providers when
+but you can add ``--install-providers-from-sources=true`` flag to Breeze to skip installing providers when


Shouldn't it be True?

Dockls are not yet updated. I leave it for the switch :) (next PR)

There will be quite many changes in the docs :)

github-actions · 2022-04-03T18:47:31Z

The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.

This is a review and clean-up for all the parameters and commands for Breeze2 in order to prepare it for being used by the contribugors. There are various small fixes here and there, removal of duplicated code, refactoring and moving code around as well as cleanup and review all the parameters used for all implemented commands. The parameters, default values and their behaviours were updated to match "new" life of Breeze rather than old one. Some improvements are made to the autocomplete and click help messages printed. Full list of choices is always displayed, parameters are groups according to their target audience, and they were sorted according to importance and frequency of use. Various messages have been colourised according to their meaning - warnings as yellow, errors as red and informational messages as bright_blue. The `dry-run` option has been added to just show what would have been run without actually running some potentially "write" commands (read commands are still executed) so that you can easily verify and manually copy and execute the commands with option to modify them before. The `dry_run` and `verbose` options are now used for all commands. The "main" command now runs "shell" by default similarly as the original Breeze. All "shortcut" parameters have been standardized - i.e common options (verbose/dry run/help) have one and all common flags that are likely to be used often have an assigned shortcute. The "stop" and "cleanup" command have been added as they are necessary for average user to complete the regular usage cycle. Documentation for all the important methods have been updated.

edithturn · 2022-04-03T23:59:32Z

wow wow, Breeze looks really amazing, @potiuk 👏🏼
Congratulations!

Bowrna · 2022-04-04T04:28:33Z

dev/breeze/src/airflow_breeze/breeze.py

+            },
+        ]
+    }
+


Woah this is really cool :)

potiuk · 2022-04-04T04:45:15Z

Indeed. This week we are going to switch everyone to it :)

potiuk · 2022-04-04T04:48:19Z

And the code is now structured for implementing the rest much better :). I had a lot of fun refactoring and moving parts (with the help of IntelliJ' refactoring features) and it was already very clean and well structured so I mostly just shuffeld things around, made some cleanups and common code extraction and clarified the use of some parameters (removed some of them or simplified them a little. That was mostly it :)

potiuk · 2022-04-04T04:49:38Z

Please take a look at the structure and docs I added - to familarise a bit with the new structure (thought the code is mostly the same, just you will find it in new places sometimes ;)

potiuk requested review from ashb, bbovenzi, jedcunningham, kaxil, mik-laj and ryanahamilton as code owners April 3, 2022 17:57

boring-cyborg bot added area:dev-tools area:webserver Webserver related Issues labels Apr 3, 2022

potiuk requested a review from eladkal April 3, 2022 17:57

eladkal approved these changes Apr 3, 2022

View reviewed changes

github-actions bot added the full tests needed We need to run full set of tests for this PR to merge label Apr 3, 2022

potiuk force-pushed the add-possible-parameters branch from 7b1d0b3 to c0645d9 Compare April 3, 2022 18:53

potiuk force-pushed the add-possible-parameters branch from c0645d9 to 364f2aa Compare April 3, 2022 19:15

potiuk merged commit 4ffd4f0 into apache:main Apr 3, 2022

potiuk deleted the add-possible-parameters branch April 3, 2022 20:35

Bowrna reviewed Apr 4, 2022

View reviewed changes

dev/breeze/src/airflow_breeze/breeze.py

},

]

}

Copy link

Contributor

Bowrna Apr 4, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Woah this is really cool :)

potiuk reacted with rocket emoji

ephraimbuddy added the changelog:skip Changes that should be skipped from the changelog (CI, tests, etc..) label Apr 14, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Prepare Breeze2 for prime time :) #22713

Prepare Breeze2 for prime time :) #22713

potiuk commented Apr 3, 2022

Uh oh!

potiuk commented Apr 3, 2022

Uh oh!

potiuk commented Apr 3, 2022

Uh oh!

eladkal left a comment

Uh oh!

eladkal Apr 3, 2022

Uh oh!

potiuk Apr 3, 2022 •

edited

Loading

Uh oh!

potiuk Apr 3, 2022

Uh oh!

github-actions bot commented Apr 3, 2022

Uh oh!

edithturn commented Apr 3, 2022

Uh oh!

Bowrna Apr 4, 2022

Uh oh!

potiuk commented Apr 4, 2022

Uh oh!

potiuk commented Apr 4, 2022

Uh oh!

potiuk commented Apr 4, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Prepare Breeze2 for prime time :) #22713

Prepare Breeze2 for prime time :) #22713

Conversation

potiuk commented Apr 3, 2022

Uh oh!

potiuk commented Apr 3, 2022

Uh oh!

potiuk commented Apr 3, 2022

Uh oh!

eladkal left a comment

Choose a reason for hiding this comment

Uh oh!

eladkal Apr 3, 2022

Choose a reason for hiding this comment

Uh oh!

potiuk Apr 3, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

potiuk Apr 3, 2022

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Apr 3, 2022

Uh oh!

edithturn commented Apr 3, 2022

Uh oh!

Bowrna Apr 4, 2022

Choose a reason for hiding this comment

Uh oh!

potiuk commented Apr 4, 2022

Uh oh!

potiuk commented Apr 4, 2022

Uh oh!

potiuk commented Apr 4, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

potiuk Apr 3, 2022 •

edited

Loading