Skip to content

Dataset changes (edit/publish) can fail due to background indexing? #7637

@qqmyers

Description

@qqmyers

It appears that API calls to :publish can fail with an OptimisticLockException when trying to set a lock (e.g. for a prepublish workflow) or later on (e.g. in a workflow step). The latter is more problematic as the workflow is asynchronous, so there's no sign in the reponse to the publish api call that the workflow fails (any may not even get to the point of pinging an external tool with the http/sr or new http/authext steps.

The best theory so far is that this can happen if indexing, running in the background, happens to update the indextime and permissionindextime on the dataset dvobject while the API call/workflow are modifying the dataset.

As a partial fix/workaround, I've tried adding a parameter to :publish to have it fail with a 409/conflict is indexing is still tbd (along with an api call to see the various modification times). I'll put in a draft PR on this.

More broadly, if I understand correctly, indexing doesn't respect locks, so it could potentially interfere with other dataset changes (when a sequence of changes is made and subsequent changes are coming fast enough to overlap with indexing from earlier changes). It seems that either having it respect locks, or a perhaps a redesign to put the index times in a separate object might be better.

(It's also possible that other discussions of transaction discussions/changes could impact this issue).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions