Refactor evaluation of inbound criteria for join task #194
Conversation
Add commands to Makefile to run tox checks to simplify running tests and checks.
Run `make schemas` command and update the JSON schemas on the specs for workflow definition.
Given one or more tasks that joins on complete but then some of the tasks failed, the workflow state machine determines the workflow failed because the inbound criteria for the join task is not satisfied and there are no more tasks to run. An option is added to skip evaluation of the inbound criteria for the join task during evaluation in the workflow state machine. The workflow state machine will identify that there is a potential next task to run and keep the workflow state as running.
Refactor evaluation of inbound criteria for join task on inbound task completion and not on inbound task transition. This allows for different use cases such as multiple task transition from the same inbound task, inbound task transition on completion or inbound task transition on failure. Include additional check in the state machine to detect case where there is an unreachable join task where inbound criteria is partially satisfied but all inbound tasks have already completed and it will never be satisified.
Add more advance test cases of join all and join count. Identified a bug related to inbound branches (failure at upstream tasks) not reaching the join task.
Update the docs section on defining join task to include the alternate use case of `join: <integer>`.
Codecov Report
@@ Coverage Diff @@
## master #194 +/- ##
==========================================
- Coverage 94.02% 94.02% -0.01%
==========================================
Files 41 41
Lines 2696 2728 +32
Branches 530 540 +10
==========================================
+ Hits 2535 2565 +30
- Misses 99 100 +1
- Partials 62 63 +1
Continue to review full report at Codecov.
|
blag
left a comment
blag
left a comment
There was a problem hiding this comment.
Solid PR, just some small PEP 8 changes to make, and some smaller, optional cosmetic changes that would be nice (IMHO).
docs/source/languages/orquesta.rst
Outdated
| | | | ||
| +-- parallel_task_3 --+ | ||
| | | ||
| +-- barrier_task (only requires 2 of 3 tasks) --+ |
There was a problem hiding this comment.
I would draw this out like this:
.. code-block:: none
=---- time (not to scale) ---->
setup_task --+
|
+-- parallel_task_1 -------------------+
|
+-- parallel_task_2 --+
| |
+-- parallel_task_3 --+
|
+-- barrier_task (only requires 2 of 3 tasks) --+
|
+-- [finish]
That way it doesn't look like the join action is waiting on parallel_task_1.
There was a problem hiding this comment.
I disagree. This is not accurate description of the workflow. This shows that the parallel_task_1 is on its own and does not join with the other parallel tasks at the barrier_task. But in fact, the join is satisfied as long as 2 out of 3 parallel tasks complete regardless of which one.
There was a problem hiding this comment.
What about something like this?
.. code-block:: none
=---- time (not to scale) ---->
setup_task --+
|
+-- parallel_task_1 -------------------+
| . (joins to barrier_task)
+-- parallel_task_2 --+ .
| | .
+-- parallel_task_3 --+ .
| .
+-- barrier_task (only requires 2 of 3 tasks) --+
|
+-- [finish]
I think we should make it visually apparent that parallel_task_1 is started with the other parallel tasks, but since it isn't finished before the other two parallel tasks finish, it doesn't join to the barrier_task with the other parallel tasks before that gets scheduled.
I use the dots to indicate that it joins barrier_task but isn't required. How does that look to you?
There was a problem hiding this comment.
We talked on Slack and compromised on using * instead of | to indicate the transition to barrier_task to differentiate this join: 2 case from the join: all case.
There was a problem hiding this comment.
Ok I changed to all asterisk for all lines from parallel tasks to the barrier task. Let's hope balance between accuracy and descriptive picture is good enough.
Refactor evaluation of inbound criteria for join task on inbound task completion and not on inbound task transition. This allows for different use cases such as multiple task transition from the same inbound task, inbound task transition on completion or inbound task transition on failure. Include additional check in the state machine to detect case where there is an unreachable join task where inbound criteria is partially satisfied but all inbound tasks have already completed and it will never be satisfied.
Fixes #123
Fixes #190