Description
I have been working on a proposal to generate tasks dynamically for argo dsl.
I got to a point where I think I should get your feedback about whether it is a feasible design.
Currently, this is a work in progress, and I am still picking up the metaprogramming mechanics implemented in argo dsl. I wouldn't be surprised if my design isn't actually feasible and needed to be abandoned, but I did my best in terms of standing from a data science perspective that have authored airflow pipelines.
The proposal is here: https://gist.github.com/binarycrayon/75af90c1cdf660333f9903cd5822245d
Please let me know what you think!
Additional context
This has originally sent as email, and forwarded here per Marek's suggestion. I realize that argo also have a built in with_items or with_params https://github.com/argoproj-labs/argo-client-python/search?q=with_items&unscoped_q=with_items
Technically I'm no longer blocked for what I need to do, but I still want to get feedback
Description
I have been working on a proposal to generate tasks dynamically for argo dsl.
I got to a point where I think I should get your feedback about whether it is a feasible design.
Currently, this is a work in progress, and I am still picking up the metaprogramming mechanics implemented in argo dsl. I wouldn't be surprised if my design isn't actually feasible and needed to be abandoned, but I did my best in terms of standing from a data science perspective that have authored airflow pipelines.
The proposal is here: https://gist.github.com/binarycrayon/75af90c1cdf660333f9903cd5822245d
Please let me know what you think!
Additional context
This has originally sent as email, and forwarded here per Marek's suggestion. I realize that argo also have a built in
with_itemsorwith_paramshttps://github.com/argoproj-labs/argo-client-python/search?q=with_items&unscoped_q=with_itemsTechnically I'm no longer blocked for what I need to do, but I still want to get feedback