Prioritize task execution by recipe order#4
Merged
mattiarighi merged 11 commits intodevelopmentfrom Sep 23, 2019
Merged
Conversation
Contributor
|
That would be great! |
Contributor
|
Is this still a draft or can I already try it? This feature would help me a lot to get more robust number for the CPU time, which I want to report in the paper. |
Member
Author
|
You can already try it, i think it works, but I need to change it a bit so the priority of the input variables used for derivation is correctly set and I need to add unit tests. |
Contributor
|
I finally found some time to test this. Unfortunately the NCL scripts are crashing due to missing That's strange, since the |
Contributor
|
Anything I can do to help with this? |
Contributor
|
To fully make use of this feature, it would be useful to write the time required by each task in the debug log (this is done only for the provenance at the moment). In this way, the user can execute the recipe once and then reorder the tasks according to their runtimes. Is that possible? |
Contributor
|
I tried to reorder the diagnostics in The task are now started in the order listed in the recipe, but this helps only for the preprocessor part of the diagnostics. See the list below: the two tasks to derive the variable So this PR helps a lot and should be merged, but it's probably not addressing #3 completely. Without task priority: With task priority: |
Member
Author
|
I added the run time and task priority to the info message, this should help with the ordering.
This is because the current implementation should start as many tasks as configured in config-user.yml, starting with the tasks with the lowest priority. However, it can only start a task if all of it's ancestors are ready. Could you post the results above, but including the task completed messages? Because then we can see if there is a problem or tasks are simply waiting for ancestor tasks. |
Contributor
I think they are simply waiting for ancestors: This is a nice example, since |
Member
Author
In that case, I don't think there is any way in which changing the order could help any further with reducing runtime. Or do you have ideas? |
Contributor
No, but I wonder why we don't get the 68 tasks of this recipe always executed in the same order. This is the starting order of the tasks in the 5 tests I've run with task1.txt The first 16 in the list are basically always the same (since I run with 16 parallel tasks). But afterwards the order changes quite randomly. For example, the bottleneck task Maybe I should try to look at the other tasks as well and try to optimize the order further. Should we mention this new feature somewhere in the documentation? |
Member
Author
That is to be expected, since they will be the highest priority tasks without ancestors, this is always the same
This is probably due to variation in the run-time of the ancestor tasks. This variation can be caused by waiting for a shared resource. This could be lots of things, it could be other preprocessing tasks if you're running too many tasks on the same node, or it could be access to storage affected by other user's demands...
Yes, that might help, but it really depends on what is causing the variation.
Good point, I'll add documentation to this pull request. |
Contributor
|
Thanks for clarifying! |
Member
Author
|
@mattiarighi I ended up adding a lot more documentation than planned, because it looked like the concept of tasks was not explained anywhere, so it was difficult to talk about their priority. |
mattiarighi
reviewed
Sep 23, 2019
mattiarighi
reviewed
Sep 23, 2019
mattiarighi
reviewed
Sep 23, 2019
mattiarighi
reviewed
Sep 23, 2019
mattiarighi
reviewed
Sep 23, 2019
Contributor
Well done, thanks! |
Member
Author
Changed the text accordingly |
mattiarighi
approved these changes
Sep 23, 2019
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #3