Skip to content

Conversation

@jedcunningham
Copy link
Member

@jedcunningham jedcunningham commented May 9, 2023

If you run a task via the cli with cfg_path, we also run configure_vars again so that the globals built from the config file can be updated. We were also initializing pluggy in that function, however, which was effectively removing any user provided hooks.

We will move the initializing of pluggy out of that function, making it safe to run again by the task command.

This primarily impacted KPOs, as they wouldn't have been passed through any pod_mutation_hooks.

If you run a task via the cli with `cfg_path`, we also run
`configure_vars` again so that the globals built from the config file
can be updated. We were also initializing pluggy in that function,
however, which was effectively removing any user provided hooks.

We will move the initializing of pluggy out of that function, making it
safe to run again by the task command.
@jedcunningham jedcunningham added this to the Airflow 2.6.1 milestone May 9, 2023
@jedcunningham
Copy link
Member Author

I decided not to write a test for this, I think the likelihood of a regression here is pretty small.

@ashb ashb changed the title Don't reinitialize pluggy when running tasks with cfg_path Ensure the KPO runs pod mutations hook correctly May 9, 2023
@jedcunningham jedcunningham merged commit 71c2627 into apache:main May 9, 2023
@jedcunningham jedcunningham deleted the fix_policy_functions_during_runtime branch May 9, 2023 22:59
@jedcunningham jedcunningham changed the title Ensure the KPO runs pod mutations hook correctly Ensure the KPO runs pod mutation hooks correctly May 9, 2023
@ephraimbuddy ephraimbuddy added the type:bug-fix Changelog: Bug Fixes label May 10, 2023
ephraimbuddy pushed a commit that referenced this pull request May 10, 2023
If you run a task via the cli with `cfg_path`, we also run
`configure_vars` again so that the globals built from the config file
can be updated. We were also initializing pluggy in that function,
however, which was effectively removing any user provided hooks.

We will move the initializing of pluggy out of that function, making it
safe to run again by the task command.

This primarily impacted KPOs, as they wouldn't have been passed through any `pod_mutation_hooks`.

(cherry picked from commit 71c2627)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type:bug-fix Changelog: Bug Fixes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants