Listenning to more fine-grained events on scratch build

[LecrisUT]

Description

There is a request to be able to start tmt tests as soon as possible (right after x86_64 builds). Afaict all of the components for listening to the specific taskid is already in place isn't it?

A few things to consider:

• Does koji download work on partially completed scratch builds?
• Do we wait for the tasks in both x86_64 and aarch64 (since testing-farm can be run on multiple architectures at the same time) before submitting the testing-farm job, do we have an opt-in for one or another behavior?
• Zuul's approach is to completely separate the scratch builds, and afaict it is because the design there was waiting for the koji task directly. But we shouldn't be restricted by needing a similar design if we listen to the events right?

Importance

Requested by @hroncok. Not sure how much of a blocker for the Zuul migration each request is though.

Workaround

• There is an existing workaround that can be used until this is implemented.

[hroncok]

koji download works, but you need to use the completed subtask id. If you use the parent task id, it waits for the entire build to finish.

[hroncok]

About the priority of this: it is very nice to have. Zuul provided faster feedback loop, but considering the current instability of it, it rarely finishes successfully before packit nowdays.

[lbarcziova]

So current behaviour is:

• we listen to the parent task state changes (buildsys.task.state.change)
• for tests, we do the check of the state here (all architectures complete)
• we pass the parent (whole) task ID to Testing Farm here

After checking the buildsys.task.state.change events from datagrepper, it looks like individual buildArch subtasks don't emit their own state change events - only the parent task does. I also checked other buildsys.* messages and don't see any that would fit this usecase.

This would mean we can't get event-driven notifications when individual architectures complete. We'd need to either do some kind of polling for subtask status, or change the architecture to mimic Zuul. Both options bring significant complexity. I will double check these tomorrow and discuss with the team.

[hroncok]

We'd need to either do some kind of polling for subtask status, or change the architecture to mimic Zuul.

FWIW Please don't mimic Zuul here. The "individual koji builds for each architecture" approach brings other issues and is overall not nice.

Polling seems like a more reasonable thing. This only makes a difference for long builds anyway, so the polling time window can be relatively large (say, 5 minutes?). If x86_64 takes 35 minutes and s390x takes 1:40, this can still start the tests 1 hour sooner.

[hroncok]

it looks like individual buildArch subtasks don't emit their own state change events

Perhaps this could be changed as well?

[LecrisUT]

it looks like individual buildArch subtasks don't emit their own state change events

Perhaps this could be changed as well?

Hopefully we can get this changed. I raised an issue in upstream fedora-infra/koji-fedoramessaging#155