Skip to content

Conversation

@ashb
Copy link
Member

@ashb ashb commented Feb 25, 2020

The refactor in #7488 introduced a bug.

I am not happy about merging this without a unit test to stop it breaking again, but given how much process_file does it's hard to test. Thoughts?


Issue link: AIRFLOW-6867

Make sure to mark the boxes below before creating PR: [x]

  • Description above provides context of the change
  • Commit message/PR title starts with [AIRFLOW-NNNN]. AIRFLOW-NNNN = JIRA ID*
  • Unit tests coverage for changes (not needed for documentation changes)
  • Commits follow "How to write a good git commit message"
  • Relevant documentation is updated including usage instructions.
  • I will engage committers as explained in Contribution Workflow Example.

* For document-only changes commit message can start with [AIRFLOW-XXXX].


In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.
Read the Pull Request Guidelines for more information.

The refactor in apache#7488 introduced a scheduler-breaking bug.
@boring-cyborg boring-cyborg bot added the area:Scheduler including HA (high availability) scheduler label Feb 25, 2020
@ashb ashb changed the title [AIRFLOW-6867] Fix bug in kill zombies [AIRFLOW-6867] Fix bug in how we call kill_zombies Feb 25, 2020
@ashb ashb requested review from kaxil and mik-laj February 25, 2020 13:23
self.log.exception("Error logging import errors!")
try:
self.kill_zombies(zombies)
self.kill_zombies(dagbag, zombies)
Copy link
Member

@kaxil kaxil Feb 25, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Surprised that our static checkers didn't catch this! Required arg was not passed !

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah quite!

Copy link
Member

@kaxil kaxil Feb 25, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe pylint would have caught it but it is currently in pylint_todo, so might have missed:

./airflow/jobs/scheduler_job.py

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed .

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tomek has the PR #7484 in progress for that.

@Fokko
Copy link
Contributor

Fokko commented Feb 25, 2020

Duplicate of #7531

@ashb ashb closed this Feb 25, 2020
@ashb ashb deleted the fix-kill-zombies branch February 25, 2020 14:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:Scheduler including HA (high availability) scheduler

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants