Skip to content

Add a context manager to execute_DAG and gufe's Context class #689

@ethanholz

Description

@ethanholz

Is your feature request related to a problem? Please describe.

As of today, execution of tasks has no guarantees around writing data in failed units. This should be handled.

Describe the solution you'd like

I propose that we had a new method that will both expand the functionality of unit contexts to be storage agnostic, but also provide robust failover capabilities. This will be done by rewriting the Context class to leverage the ExternalStorage for filesystem operations in shared/permanent cases. Furthermore, execute_DAG will be expanded to leverage these new primitives to provide a failure case for writing files back to storage mediums.

Describe alternatives you've considered

n/a

Additional context

This work is being done as part of exorcist to enable robust campaign-scale orchestration.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions