fix(cssc): 31731403 Allow generate_logs poller to timeout#36
Merged
cegraybl merged 6 commits intoMar 18, 2025
Merged
Conversation
There was a problem hiding this comment.
Pull Request Overview
This PR addresses a potential hang in the log generation process by adding a timeout for the task run poller, ensuring that if a task run does not complete within 10 minutes, the system logs the timeout and attempts to retrieve logs if available.
- Introduces a timeout mechanism in the WorkflowLogPollingMethod class.
- Wraps the LongRunningOperation invocation in a try/except block to capture TimeoutError.
- Enhances logging for both normal execution and error scenarios.
cegraybl
force-pushed
the
cegraybl/cssc_31731403_geeneratelogs_timeout
branch
from
2782749 to
6eea674
Compare
huanwu
approved these changes
Mar 15, 2025
cegraybl
added a commit
that referenced
this pull request
Apr 23, 2025
To retrieve task logs class `WorkflowTaskStatus` uses a Poller to wait for a task run to finish before attempting to download the logs. This is only done during image limit check or when the `--dry-run` option is used during the `create` and `update` commands. Since this is a remote call there is a chance that we never get a response. This PR adds a 10-minute timeout to the Poller. In case the timeout is reached, the required message is logged, and the attempt will be done to retrieve the logs if there are any to allow the operation to continue. In case of a timeout during `create` or `update` the command will not block in case the image limit check could not be performed, to allow for a better user experience. In case the image limit is reached, the user will be informed via the trigger task that will perform the check every time it runs.
cegraybl
added a commit
that referenced
this pull request
Apr 24, 2025
To retrieve task logs class `WorkflowTaskStatus` uses a Poller to wait for a task run to finish before attempting to download the logs. This is only done during image limit check or when the `--dry-run` option is used during the `create` and `update` commands. Since this is a remote call there is a chance that we never get a response. This PR adds a 10-minute timeout to the Poller. In case the timeout is reached, the required message is logged, and the attempt will be done to retrieve the logs if there are any to allow the operation to continue. In case of a timeout during `create` or `update` the command will not block in case the image limit check could not be performed, to allow for a better user experience. In case the image limit is reached, the user will be informed via the trigger task that will perform the check every time it runs.
mabelegba
pushed a commit
that referenced
this pull request
Nov 25, 2025
* Upstream Merge * Merge from workload-orchestration : Added Example in Description (#34) * Merge from workload-orchestration : Fixed Example * Fixed Example for SolutionTemplateVersion * Made Changes for command-change CI fix' * Reset Version 1.0.0b1 * Added Bulk and Diagnostics Back (#35) Co-authored-by: Atharva Udapure <audapure@microsoft.com> * Added Bulk Solution Example (#36) * Added Bulk and Diagnostics Back * Changes --------- Co-authored-by: Atharva Udapure <audapure@microsoft.com> * Fixed Solution Template Linter Issue * Added Integration Tests Framework for WorkloadOrchestration #37 Co-authored-by: Atharva Udapure <audapure@microsoft.com> * Added ServiceName (#38) * Added Tests * Made Changes on ServiceName --------- Co-authored-by: Atharva Udapure <audapure@microsoft.com> * Added Full Target Solution Tests (#39) * Added Tests * Made Changes on ServiceName * Nit Changes * Made Changes for ContextLookup --------- Co-authored-by: Atharva Udapure <audapure@microsoft.com> * Added E2E workflow Tests (#40) * Added Tests * Made Changes on ServiceName * Nit Changes * Made Changes for ContextLookup * Added complete workflow * Add Licence * Added License --------- Co-authored-by: Atharva Udapure <audapure@microsoft.com> * Added Readme (#41) Co-authored-by: Atharva Udapure <audapure@microsoft.com> * Bulk Deployemnent LRO changes (#42) * made changes * Added change * Added Changes in Test * Added CLi Changes * Added Setup --------- Co-authored-by: Atharva Udapure <audapure@microsoft.com> * Added Changes For Backward Compitablity (#43) * Added Changes For Backward Compitablity * Made changes --------- Co-authored-by: Atharva Udapure <audapure@microsoft.com> * Made changes (#44) Co-authored-by: Atharva Udapure <audapure@microsoft.com> * Target Operations API refractoring (#47) * Changes I made * Make changes * Made chnages * Make changes --------- Co-authored-by: Atharva Udapure <audapure@microsoft.com> * Stable API 2025-06-01 (#48) * Made changes * Made Changes --------- Co-authored-by: Atharva Udapure <audapure@microsoft.com> * Change in ID in Targets (#50) * Made changes * changes --------- Co-authored-by: Atharva Udapure <audapure@microsoft.com> * Remove Id and Name from Target Review (#51) Co-authored-by: Atharva Udapure <audapure@microsoft.com> * final bulk (#52) Co-authored-by: Atharva Udapure <audapure@microsoft.com> * Changes in ARG * Added Linter Exception * Added ITTests (#53) Co-authored-by: Atharva Udapure <audapure@microsoft.com> * Standardize CLI (#54) * Made Changes * Made Changes * Made changes * Added Change * Made changes * Made changes * Made changes * Made Changes --------- Co-authored-by: Atharva Udapure <audapure@microsoft.com> * Linter FIxes and Remove Preview * Removed Resolved from CLI * change in version * made changes (#55) Co-authored-by: Atharva Udapure <audapure@microsoft.com> * Fixed history --------- Co-authored-by: Atharva Udapure <audapure@microsoft.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
To retrieve task logs class
WorkflowTaskStatususes a Poller to wait for a task run to finish before attempting to download the logs. This is only done during image limit check or when the--dry-runoption is used during thecreateandupdatecommands. Since this is a remote call there is a chance that we never get a response.This PR adds a 10-minute timeout to the Poller. In case the timeout is reached, the required message is logged, and the attempt will be done to retrieve the logs if there are any to allow the operation to continue.
In case of a timeout during
createorupdatethe command will not block in case the image limit check could not be performed, to allow for a better user experience. In case the image limit is reached, the user will be informed via the trigger task that will perform the check every time it runs.