Ability to Restart Graph From Failure #574
Description
What
If a particular collection fails during an access or an erasure request, raise an exception and cancel other tasks in the graph. Allow the privacy request to be restarted from the failure point.
Why
We currently retry a failed collection a specified number of times. If the collection continues to fail after a certain number of retries, we continue with the graph execution. We assume that the failed collection didn't return data, and that no data was masked. Downstream collections still run.
This can be problematic for both data in the failed collection and downstream collections whose data potentially wasn't retrieved or masked. Running another privacy request may not rectify the issue because data may have been destroyed that prevents us from reaching the collection in question again.
We will still keep the retry, in case the failure was just temporary, but stopping execution entirely allows the user to go correct something on their end before resuming.