API Endpoints: Update Metadata For a Dataset / Edit Dataset Metadata

[transfluxus]

What steps does it take to reproduce the issue?
It needs a published dataset.
The 2 endpoints for updating the metadata fail:
/api/datasets/:persistentId/versions/:latest?persistentId=$PERSISTENT_IDENTIFIER results in a
400, {"status":"ERROR","message":"Only the :draft version can be updated"}

/api/datasets/:persistentId/editMetadata/?persistentId=$PERSISTENT_IDENTIFIER
OR
api/datasets/:persistentId/editMetadata/?persistentId=doi:10.5072/FK2/BCCP9Z&replace=true
result in responses with status code 500.

• When does this issue occur?
  All the time, for published datasets, but it the endpoint for drafts works

• Which page(s) does it occurs on?
  demo.dataverse.org

• What happens?

• To whom does it occur (all users, curators, superusers)?
  me the developer

• What did you expect to happen?
  A 200, with updated metadata

Which version of Dataverse are you using?
v. 5.12.1 build 1122-cf90431

Any related open or closed issues to this bug report?

Screenshots:

No matter the issue, screenshots are always welcome.

To add a screenshot, please use one of the following formats and/or methods described here:

• https://help.github.com/en/articles/file-attachments-on-issues-and-pull-requests

[qqmyers]

A quick partial answer:

FWIW: /api/datasets/:persistentId/versions/:draft should work even if there isn't a draft version currently. It should create one. (Not sure why the API call allows you to specify which version when only :draft is allowed).

For the editMetadata calls - those are tested in the automated test suite so it would be a little surprising if they fail in general. A more likely scenario would be that a 500 error is being returned when a 400/Bad request response would be better. Can you provide more detail on what you are sending in those requests?

[transfluxus]

Hi qqmyers

thanks, I thought that there would be tests.

After getting reading the metadata of a dataset and storing into a json I create a new json, just copying out the metadataBlocks:

{
  "metadataBlocks": {
        "citation": {
          "displayName": "Citation Metadata",
          "name": "citation",
          "fields": [
            {
              "typeName": "title",
              "multiple": false,
              "typeClass": "primitive",
              "value": "First publication"
            },
            {
              "typeName": "author",
              "multiple": true,
              "typeClass": "compound",
              "value": [
                {
                  "authorName": {
                    "typeName": "authorName",
                    "multiple": false,
                    "typeClass": "primitive",
                    "value": "s, ramin"
                  },
                  "authorAffiliation": {
                    "typeName": "authorAffiliation",
                    "multiple": false,
                    "typeClass": "primitive",
                    "value": "uab"
                  }
                }
              ]
            },
            {
              "typeName": "datasetContact",
              "multiple": true,
              "typeClass": "compound",
              "value": [
                {
                  "datasetContactName": {
                    "typeName": "datasetContactName",
                    "multiple": false,
                    "typeClass": "primitive",
                    "value": "s, ramin"
                  },
                  "datasetContactAffiliation": {
                    "typeName": "datasetContactAffiliation",
                    "multiple": false,
                    "typeClass": "primitive",
                    "value": "uab"
                  },
                  "datasetContactEmail": {
                    "typeName": "datasetContactEmail",
                    "multiple": false,
                    "typeClass": "primitive",
                    "value": "ramin@uab.cat"
                  }
                }
              ]
            },
            {
              "typeName": "dsDescription",
              "multiple": true,
              "typeClass": "compound",
              "value": [
                {
                  "dsDescriptionValue": {
                    "typeName": "dsDescriptionValue",
                    "multiple": false,
                    "typeClass": "primitive",
                    "value": "number 1"
                  }
                }
              ]
            },
            {
              "typeName": "subject",
              "multiple": true,
              "typeClass": "controlledVocabulary",
              "value": [
                "Earth and Environmental Sciences"
              ]
            },
            {
              "typeName": "depositor",
              "multiple": false,
              "typeClass": "primitive",
              "value": "s, ramin"
            },
            {
              "typeName": "dateOfDeposit",
              "multiple": false,
              "typeClass": "primitive",
              "value": "2023-01-18"
            }
          ]
        }
      }
}

the error messages now seem a bit like a plotwist!
"Validation Failed" and "Only the :draft version can be update"

export API_TOKEN=
export SERVER_URL=https://demo.dataverse.org
# published dataset
export PERSISTENT_IDENTIFIER=doi:10.70122/FK2/DKNBH5

# READ
#curl -H "X-Dataverse-key:$API_TOKEN" $SERVER_URL/api/datasets/:persistentId/?persistentId=$PERSISTENT_IDENTIFIER -o dataset-complete.json


# UPDATE DRAFT
curl -H "X-Dataverse-key: $API_TOKEN" -X PUT $SERVER_URL/api/datasets/:persistentId/versions/:draft?persistentId=$PERSISTENT_IDENTIFIER --upload-file dataset-update-metadata.json

> {"status":"ERROR","message":"Validation Failed: Failed Validation Terms Of Use and Access (Invalid value:edu.harvard.iq.dataverse.TermsOfUseAndAccess[ id=null ]).java.util.stream.ReferencePipeline$3@4e9d4b30"}{


# UPDATE
curl -H "X-Dataverse-key: $API_TOKEN" $SERVER_URL/api/datasets/:persistentId/versions/:latest?persistentId=$PERSISTENT_IDENTIFIER --upload-file dataset-update-metadata.json

> {"status":"ERROR","message":"Only the :draft version can be updated"}

[pdurbin]

{"status":"ERROR","message":"Validation Failed: Failed Validation Terms Of Use and Access (Invalid value:edu.harvard.iq.dataverse.TermsOfUseAndAccess[ id=null ]).java.util.stream.ReferencePipeline$3@4e9d4b30"}{

This reminds me of the 5.11 release notes where we wrote this:

The API Guide has been updated to explain that the Content-type header is now (as of Dataverse 5.6) necessary to create datasets via native API. (Issue #8663, PR #8676)

@transfluxus can you please try adding -H 'Content-type:application/json' to your curl command?

Just a guess! It may not help! 😅 Thanks for providing the JSON! Yes, we should add a test for this.

[transfluxus]

Hi,

I changed the command to:

curl -H "X-Dataverse-key: $API_TOKEN" -H 'Content-type:application/json' $SERVER_URL/api/datasets/:persistentId/versions/:latest?persistentId=$PERSISTENT_IDENTIFIER --upload-file dataset-update-metadata.json

and still get: {"status":"ERROR","message":"Only the :draft version can be updated"}

[pdurbin]

Bah, ok, thanks for trying, @transfluxus!

[qqmyers]

Anything with :latest will definitely not work. Using :draft should work both when a draft exists and when one needs to be created to make your changes. (Adding the Content-type as Phil said may also be needed, I'm not sure).

In any case, the new error you're getting when you use :draft seems to be related to another section in those 5.11.1 release notes: Terms of Access or Request Access Required for Restricted Files. It says there that "Datasets that are in draft and that have neither Terms of Access nor Request Access enabled must be updated to select one or the other (or both). Otherwise, datasets cannot be further edited or published." I believe trying to edit a draft version created prior to 5.11 that didn't have request access enabled and had no terms of access via the API call your using would give the error you see. The release notes have more detail. Hopefully this is the case and if you manually update the terms for that dataset version or test with a new dataset, things should work. (If you have many dataset versions in this state, the release notes has info on how to query to find/fix them.)

[transfluxus]

Hi,
After some absence on this I tried it again. I could update a published dataset, which was my goal.
Maybe the docs on updating (https://guides.dataverse.org/en/5.12.1/api/native-api.html#update-metadata-for-a-dataset
) should mention that next to the metadataBlocks some other values might be required, in my case e.g.termsOfUse
and termsOfAccess needed to be included because of custom terms and restricted files.

[pdurbin]

@transfluxus good idea. Would you like to make a pull request? The file to edit is here: https://github.com/IQSS/dataverse/blob/develop/doc/sphinx-guides/source/api/native-api.rst

Here is our "quick fix" guide: https://guides.dataverse.org/en/5.13/developers/documentation.html#quick-fix

[transfluxus]

I could do that. But are there other fields, that I didn't encounter, that would need to be included when there are present?

[pdurbin]

Hmm, that's quite possible, yes. 🤔