[Datastore Management] Disable/Delete datastore BACKEND

[adriaaaa]

Is your feature request related to a specific problem?

Organizations must be able to delete a datastore connection (in the event that they stop using a service or deprecate a db) and disable a datastore connection (to allow for SR processing but to temporarily disable for troubleshooting purposes).

Additional Context

• A datastore consists of dataset config + connection config. Deleting a datastore implies we are deleting both the dataset config and the connection config.
• Reqs

Requirements/Acceptance Criteria

1. Confirm that we log when and who deleted or disabled a datastore
2. Demonstrate that the following behavior occurs when attempting to DELETE a datastore:

Scenario	Expected behavior
The Subject Request is in the NEW state, a connected datastore was deleted, then the Subject Request is approved	The Subject Request will begin processing and should not try to include the deleted datastore in it’s workflow (since it was deleted).
The Subject Request is in the IN PROGRESS state and then a connected datastore was deleted	The Subject Request should continue processing and try to access that datastore. If it cannot, then the request should fail/error and log the reason why it failed. If it was successful, then the information from that datastore should be included (in the event of an access request) or erased (in the event of an erasure request)
The Subject request is in an ERRORED state, the datastore that was previously connected and enabled is now deleted, and then someone went to reprocess the request.	If the subject request is reprocessed by a user, then the workflow should not try to connect to the deleted datastore (since it has been deleted and should no longer be part of the workflow)
The Subject Request is in the COMPLETED state, a connected datastore was deleted	The Subject Request workflow should have already attempted to connect to that datastore and was either completed successfully or errored in a non-fatal way. The datastore status should remain on the activity timeline (execution logs) for that subject request
The Subject Request is in the DENIED state, a connected datastore was deleted	Since the Subject Request was denied, it should not have attempted to reach out to any datastores, so there should be no chance in the functionality here.

4. Demonstrate that the following behavior occurs when attempting to DISABLE a datastore

Scenario	Expected behavior
The Subject Request is in the NEW state, a connected datastore was disabled, then the Subject Request is approved	The Subject Request should not attempt to connect to that disabled datastoresThe execution logs should include the datastore in the logs but mention that it was disabled so fidesops did not attempt to connect to it.
The Subject Request is in the IN PROGRESS state and then a connected datastore was disabled	The Subject Request should continue processing and should try to continue to access that datastore.
The Subject request is in an ERRORED state, the datastore that was previously connected and enabled is now disabled, and then someone went to reprocess the request.	The subject request should start reprocessing the request and SKIP the disabled datastore in it’s workflow.
The Subject Request is in the COMPLETED state, a connected datastore was disabled, then the Subject Request is approved	The Subject Request workflow should have already attempted to connect to that datastore and was either completed successfully or errored in a non-fatal way. The datastore should remain on the activity timeline (execution logs) for that subject request with the appropriate datastore status.
The Subject Request is in the DENIED state, a connected datastore was disabled, then the Subject Request is approved	Since the Subject Request was denied, it should not have attempted to reach out to any datastores, so there should be no chance in the functionality here.

[pattisdr]

I believe that the delete behavior is already supported by default, so we'd just need to verify, and add a few small tweaks for disable, as this is a new concept:

1. Verify that DELETE behavior is as expected (in progress is the main one to double check)
   a) NEW request should already build graph from scratch, and the graph wouldn't have deleted dataset configs.
   b) IN PROGRESS request should have already built the graph, so the deleted datastore will be in the graph already. I believe execution would continue as normal because it's in memory. If something does fail, Restart Graph from Failure [#574] #578 now exits graph execution if there's a failure, cancelling remaining tasks.
   c) COMPLETE request - this request is in the past. Nothing would change, and we wouldn't delete existing execution logs from obsolete collections.
   d) DENIED request - this request is in the past. Nothing would change.
   e) ERRORED - on retry, the new graph should not have the deleted datastore
2. Add a field on the ConnectionConfig model store if it's disabled, and its disabled_date. Dataset configs should automatically inherit their ConnectionConfig's disabled state.
3. Allow updating the disabled field in PATCH /connectionconfig
4. Add a method to GraphTask that gets the ConnectionConfig for the given node from memory (similar to how we set GraphTask.connector. GraphTask.access_request and GraphTask.erasure_request should both verify that their ConnectionConfigs are enabled before proceeding. If not, create a skipped execution log, and pass an empty list (for access) or 0 (for erasure) downstream instead of attempting to run any of the normal logic.
   a) This will create the expected behavior for "new" and "restart from failure" disabled requests.
5. Verify DISABLE behavior