Support `-Timeout` for test cases

[SteveL-MSFT]

1. Provide a general summary of the issue in the Title above

For PSCore6, we've had to create a custom test function so that tests don't hang during execution (eventually timing out in CI).

It would be ideal if It had a parameter to specify -Timeoutms and fail the test case.

2. Describe Your Environment

Operating System, Pester version and PowerShell version:

Pester version     : 4.1.1 /Users/steve/.local/share/powershell/Modules/Pester/4.1.1/Pester.psm1
PowerShell version : 6.0.1
OS version         : Unix 17.4.0.0

3. Expected Behavior

It "Timeout test" -Timeoutms 500 {
  Start-Sleep -Seconds 1
}

# this should result in the test failing due to timeout, aftereach/afterall should still be called, applies to individual test cases

4.Current Behavior

Need to implement custom timeout function and use it everywhere applicable

5. Possible Solution

In PSCore6.1 we may be able to add -Timeout to Invoke-Command, but for compatibility reasons, you may use something like the custom function we have above.

6. Context

Usually webcmdlet tests that need to call external URLs cause problems if the web server doesn't respond. Although the webcmdlets themselves have a timeout, this affects the code logic and what is actually being tested.

[iSazonov]

Maybe we need a global default value.

[SteveL-MSFT]

@iSazonov perhaps have it for Describe as well?

[iSazonov]

This is expected.

[nohwnd]

@SteveL-MSFT
Thanks for the suggestion, but I don't think your spin wait works. The call to .Invoke() is synchronous (tested on PS and PSCore), so your while is just waiting in the condition for the scriptblock to finish, without actually running the body of the loop.

$sb = { 
    $s = 1..10 | %{ Get-Service }
    write-host "sb done" 
}
write-host "start"
$sb.Invoke()
write-host "end"

#output
start
sb done
end

function Wait-UntilTrue
{
    [CmdletBinding()]
    param (
        [ScriptBlock]$sb,
        [int]$TimeoutInMilliseconds = 10000,
        [int]$IntervalInMilliseconds = 1000
        )
    # Get the current time
    $startTime = [DateTime]::Now

    # Loop until the script block evaluates to true
    while (-not ($sb.Invoke())) {
        # If the timeout period has passed, return false
        $passedTime = ([DateTime]::Now - $startTime).TotalMilliseconds
        Write-Host "sb it is not done yet $passedTime"
        if ($passedTime -gt $timeoutInMilliseconds) {
            return $false
        }
        # Sleep for the specified interval
        Start-Sleep -Milliseconds $intervalInMilliseconds
    }
    return $true
}

$sb = {  
    $s = 1..10 | %{ Get-Service }
    write-host "sb done" 
}
Wait-UntilTrue $sb -TimeoutInMilliseconds 1000 -IntervalInMilliseconds 100

# output 
sb done
sb it is not done yet 2930.5421
False

Is there any simple way of timing out the script block, without running it asynchronously? Timers maybe?

[nohwnd]

@SteveL-MSFT if you have any proposals how to implement this please let me know and we can re-open this. I don't know about a way that would allow me to run code and pre-emptively cancel it without moving it into a runspace/job. Runinng it in the same thread does not work because it runs synchronously as shown in the previous post. Maybe I am missing something?

[iSazonov]

Currently we have a PR to add some async APIs.