-
Notifications
You must be signed in to change notification settings - Fork 16.4k
Nicer handling of cached Breeze parameters #23195
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
This has been bothered me a bit - our implementation of parameter caching with synchronization was a bit convoluted (mostly because it was based on the Bash implementation). I took a stab at it and it turned out that we can rather easily do it "properly" i.e. extending Click @Bowrna -> you might want to take a look and see that this is a really nice improvement :). @blag -> you might also take some inspiration when convert Airlfow CLI. The Click + Rich Click combination is awesome and wonderfully and easily extensible. |
e1d267a to
39e9bad
Compare
dev/breeze/tests/test_build_image.py
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Those tests can be deleted now. We completely decoupled caching parameters from creating BuildProdParams, BuildCIParams - by the time they get created, the cache retrieval/writing is already processed via CacheableChoice and those tests make no sense.
39e9bad to
330ef05
Compare
330ef05 to
993a0d0
Compare
The cached parameters in Python Breeze were largely based on Bash implementation. They did the job but required pretty cumbersome synchronization of cached values with parameters passed and it was easy to forget about this as you had to do it sepearately in each method that had potentially cacheable parameters. In this PR we take advantage of the Click class hiarchy and their extendability. We've already extended Click Choice parameter to be much better formatted for long list of choices but we take this a bit further now with adding new type of parameter that can cache the values between runs. This has multiple advantages: * we do not have to remember about synchroniation - parameters automatically read/write their values from cache as they are used * we can automatically set parameters to default when wrong value is passed. This is nice as user does not have to re-run the command because the values are corrected on-the-flight. * the parameters can have their default values displayed in help screen (we use sentinel default that we can use to detect if value was passed from parameter or taken from default) * the parameters can also have their <current> values marked in list of choices - so the user in the help screen can see not only the default but also which value is currently selected and will be used if you do not pass any parameter.
993a0d0 to
fa4866e
Compare
|
merged as part of #23205 |
|
Already merged as part of #23205 |
The cached parameters in Python Breeze were largely based on
Bash implementation. They did the job but required pretty
cumbersome synchronization of cached values with parameters
passed and it was easy to forget about this as you had to
do it sepearately in each method that had potentially
cacheable parameters.
In this PR we take advantage of the Click class hiarchy and their
extendability. We've already extended Click Choice parameter
to be much better formatted for long list of choices but we take
this a bit further now with adding new type of parameter that
can cache the values between runs.
This has multiple advantages:
automatically read/write their values from cache as they are
used
value is passed. This is nice as user does not have to
re-run the command because the values are corrected
on-the-flight.
help screen (we use sentinel default that we can use to
detect if value was passed from parameter or taken from default)
list of choices - so the user in the help screen can see not
only the default but also which value is currently selected
and will be used if you do not pass any parameter.
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragement file, named
{pr_number}.significant.rst, in newsfragments.