$ dapr workflow CLI docs

[JoshVanL]

Based on dapr/cli#1560

[cicoyle]

content lgtm, just one question - might just be a format question on my end bc I didnt directly pull it down locally

[cicoyle]

should this be in a diff file or is this intentional to have in this file?

[JoshVanL]

Thanks! Should be fixed now 🙂

[msfussell]

Great docs. Primary feedback is getting these CLI commands to work with the local Redis actor store so that people can try this with the quickstarts and for the local developer experience.

[JoshVanL]

Thanks for the review @msfussell, PTAL!

[msfussell]

Few more comments based on testing the quickstart.

[msfussell]

Suggested change

	--app-id orderprocessing \
	--app-id order-processor \

[msfussell]

Also when I suspend the workflow and waited for more then 100 seconds, it would then not resume and fail

= APP - order-processor == info: System.Net.Http.HttpClient.Default.LogicalHandler[101]
== APP - order-processor ==       End processing HTTP request after 0.6682ms - 200
== APP - order-processor == Unhandled exception. System.OperationCanceledException: The WaitForInstanceCompletionAsync operation was canceled.
== APP - order-processor ==  ---> Grpc.Core.RpcException: Status(StatusCode="Cancelled", Detail="", DebugException="System.OperationCanceledException: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.")
== APP - order-processor ==  ---> System.OperationCanceledException: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
== APP - order-processor ==  ---> System.Threading.Tasks.TaskCanceledException: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
== APP - order-processor ==  ---> System.TimeoutException: The operation was canceled.
== APP - order-processor ==  ---> System.Threading.Tasks.TaskCanceledException: The operation was canceled.
== APP - order-processor ==    at System.Net.Http.Http2Connection.Http2Stream.<>c.<WaitForDataAsync>b__88_0(Object s, CancellationToken cancellationToken)
== APP - order-processor ==    at System.Threading.CancellationTokenSource.Invoke(Delegate d, Object state, CancellationTokenSource source)
== APP - order-processor ==    at System.Threading.CancellationTokenSource.ExecuteCallbackHandlers(Boolean throwOnFirstException)
== APP - order-processor ==    at System.Threading.TimerQueueTimer.Fire(Boolean isThreadPool)
== APP - order-processor ==    at System.Threading.TimerQueue.FireNextTimers()
== APP - order-processor ==    at System.Threading.ThreadPoolWorkQueue.Dispatch()
== APP - order-processor ==    at System.Threading.PortableThreadPool.WorkerThread.WorkerThreadStart()
== APP - order-processor ==    at System.Threading.Thread.StartCallback()
== APP - order-processor == --- End of stack trace from previous location ---
== APP - order-processor ==    at System.Net.Http.Http2Connection.Http2Stream.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token)
== APP - order-processor ==    at System.Net.Http.Http2Connection.Http2Stream.ReadResponseHeadersAsync(CancellationToken cancellationToken)
== APP - order-processor ==    at System.Net.Http.Http2Connection.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
== APP - order-processor ==    at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
== APP - order-processor ==    at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
== APP - order-processor ==    at Microsoft.Extensions.Http.Logging.LoggingHttpMessageHandler.<SendCoreAsync>g__Core|4_0(HttpRequestMessage request, Boolean useAsync, CancellationToken cancellationToken)
== APP - order-processor ==    at Microsoft.Extensions.Http.Logging.LoggingScopeHttpMessageHandler.<SendCoreAsync>g__Core|4_0(HttpRequestMessage request, Boolean useAsync, CancellationToken cancellationToken)
== APP - order-processor ==    at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
== APP - order-processor ==    --- End of inner exception stack trace ---
== APP - order-processor ==    --- End of inner exception stack trace ---
== APP - order-processor ==    at System.Net.Http.HttpClient.HandleFailure(Exception e, Boolean telemetryStarted, HttpResponseMessage response, CancellationTokenSource cts, CancellationToken cancellationToken, CancellationTokenSource pendingRequestsCts)
== APP - order-processor ==    at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
== APP - order-processor ==    at Grpc.Net.Client.Internal.GrpcCall`2.RunCall(HttpRequestMessage request, Nullable`1 timeout)
== APP - order-processor ==    --- End of inner exception stack trace ---
== APP - order-processor ==    at Grpc.Net.Client.Internal.GrpcCall`2.ResolveException(String summary, Exception ex, Nullable`1& status, Exception& resolvedException)
== APP - order-processor ==    at Grpc.Net.Client.Internal.GrpcCall`2.RunCall(HttpRequestMessage request, Nullable`1 timeout)

[JoshVanL]

This looks like an error coming from the application which timed out waiting for the workflow to complete-

== APP - order-processor == Unhandled exception. System.OperationCanceledException: The WaitForInstanceCompletionAsync operation was canceled.

[msfussell]

Suggested change

	dapr workflow purge order-20240312-001 --app-id orderprocessing --connection-string=redis://127.0.0.1:6379
	dapr workflow purge order-20240312-001 --app-id order-processor --connection-string=redis://127.0.0.1:6379

[msfussell]

This did not work for me. I got the following error. The workflow was in a completed state

 dapr workflow list --app-id order-processor --connection-string=redis://127.0.0.1:6379
NAME                     ID        STATUS     AGE
OrderProcessingWorkflow  a7d58b95  COMPLETED  26h

dapr workflow purge a7d58b95 --app-id order-processor --connection-string=redis://127.0.0.1:6379

ℹ️  Purging 1 workflow instance(s)
Error: a7d58b95: failed to purge orchestration state: rpc error: code = FailedPrecondition desc = failed to purge orchestration state: failed to lookup actor: api error: code = FailedPrecondition desc = did not find address for actor
Usage:
  dapr workflow purge [flags]

Flags:
      --all                          Purge all workflow instances in a terminal state. Use with caution.
      --all-older-than string        Purge workflow instances older than the specified Go duration or timestamp, e.g., '24h' or '2023-01-02T15:04:05Z'.
  -c, --connection-string string     The connection string used to connect and authenticate to the actor state store
  -h, --help                         help for purge
      --scheduler-namespace string   Kubernetes namespace where the scheduler is deployed, only relevant if --kubernetes is set (default "dapr-system")
  -t, --table-name string            The name of the table or collection which is used as the actor state store

Global Flags:
  -a, --app-id string         The app ID owner of the workflow instance
  -k, --kubernetes            Target a Kubernetes dapr installation
      --log-as-json           Log output in JSON format
  -n, --namespace string      Namespace to perform workflow operation on (default "default")
      --runtime-path string   The path to the dapr runtime installation directory

a7d58b95: failed to purge orchestration state: rpc error: code = FailedPrecondition desc = failed to purge orchestration state: failed to lookup actor: api error: code = FailedPrecondition desc = did not find address for actor

[JoshVanL]

This error happens because the workflow client is not connected. It is required that the workflow client is connected, so that the workflow actor runtime is enabled, so that the purge process goes through the same in turn lock state machine. Not doing so and purging an in progress workflow would result in corruption or some other catastrophic error.

In 1.17 we will add a --force flag to the purge CLI (and gRPC purge RPC) to allow for skipping deleting the state via the actor runtime. This needs to come with the user understanding these dangers.

I've updated the docs with following:

---

{{% alert title="Important" color="warning" %}}
In order to preserve the workflow state machine integrity and prevent corruption, purging workflow requires that the workflow client for that app ID is running.
Errors like the following suggest that the workflow client is not running:

failed to purge orchestration state: rpc error: code = FailedPrecondition desc = failed to purge orchestration state: failed to lookup actor: api error: code = FailedPrecondition desc = did not find address for actor

{{% /alert %}}

[msfussell]

I would encourage errors to be more action orientated, saying what the user can do to resolve the issue. Could we in this case say that you need to have a workflow client connected to the workflow or similar. I do find most of the errors in Dapr to be obscure in how to address them

[JoshVanL]

Updated- please let me know if that is more like how you are suggest.

[JoshVanL]

Thanks @msfussell

[msfussell]

Great to have these. Amazingly useful.