Cannot create dataset on demo server using JSON example

[pdurbin]

This issue was discovered by @cmbz but I'm writing it up after reproducing it myself.

If you follow https://guides.dataverse.org/en/5.10.1/api/native-api.html#create-a-dataset-in-a-dataverse-collection and try to use our suggested "dataset-finch1.json" file to create a dataset, it fails with the error below.

curl -H X-Dataverse-key:$API_TOKEN -X POST "$SERVER_URL/api/dataverses/$PARENT/datasets" --upload-file dataset-finch1.json

{"status":"ERROR","message":"Validation Failed: Title is required. (Invalid value:edu.harvard.iq.dataverse.DatasetField[ id=null ]), Description Text is required. (Invalid value:edu.harvard.iq.dataverse.DatasetField[ id=null ]), Subject is required. (Invalid value:edu.harvard.iq.dataverse.DatasetField[ id=null ]), Author Name is required. (Invalid value:edu.harvard.iq.dataverse.DatasetField[ id=null ]), Contact E-mail is required. (Invalid value:edu.harvard.iq.dataverse.DatasetField[ id=null ])."}

I believe this problem is limited to the demo server (though I haven't tried production) because the JSON file works fine on my laptop, which is running 92a14a8 on the develop branch. That's why I'm creating this issue here in the "operational work" issue rather than the main repo.

I'm not sure if #8533 is related but in it @kcondon reported seeing similar "Contact E-mail is required" errors (though nothing about Subject or Author Name). That issue was about the semantic API. This issue is about the regular native API.

[cmbz]

Note: although the test fails on demo.harvard.edu, it succeeds on dataverse.harvard.edu, returning:
{"status":"OK","data":{"id":6180674,"persistentId":"doi:10.7910/DVN/VM27UP"}}

[mreekie]

In production this is working, but this must be tested in "demo" environment.
The reason this is being prioritized is that this is grant funded and must be delivered for that.
The reason that this is important even though it's the "demo" environment.
large number of files in the script that is being written.
It's going to be easy to make mistakes and working in demo will allow this to be well dev tested.

[mreekie]

Sprint:

• small
• Guessing at small assuming that the demo server won't have to be taken down and rebuilt.
• It is hopeful that with some troubleshooting and comparison of the config will result in a solution.

[qqmyers]

FWIW: Adding -H 'Content-type:application/json' works to upload (with metadata properly filled in)

Note: when @qqmyers first tried it was into a dataverse that had additional (legit) requirements, so it seemed to only reduce the error, but I tried again into a simple dataverse collection and it worked.

[cmbz]

Thanks! Given this new information, I was able to create a workaround. See snippet below:

[pdurbin]

Weird! Yes, I'm seeing the same thing, adding -H 'Content-type:application/json' allows the "finch" dataset to be created:

curl -H X-Dataverse-key:$API_TOKEN -X POST "$SERVER_URL/api/dataverses/$PARENT/datasets" --upload-file dataset-finch1.json -H 'Content-type:application/json'

But why would we see this on the demo server but not prod or any other server (I can't reproduce the issue on my laptop running 7b13e75 on develop)? What's special about demo?

Seeing this reminds me that @Jeija opened this similar issue in pyDataverse about Dataverse 5.9:

• Posting JSON broken on Dataverse 5.9 gdcc/pyDataverse#143

Here's the error reported there...

{'status': 'ERROR', 'message': 'Validation Failed: Author Name is required. (Invalid value:edu.harvard.iq.dataverse.DatasetField[ id=null ]), Contact E-mail is required. (Invalid value:edu.harvard.iq.dataverse.DatasetField[ id=null ]), Description Text is required. (Invalid value:edu.harvard.iq.dataverse.DatasetField[ id=null ]), Subject is required. (Invalid value:edu.harvard.iq.dataverse.DatasetField[ id=null ]), Title is required. (Invalid value:edu.harvard.iq.dataverse.DatasetField[ id=null ]).'}

... quite similar to the one here. Interestingly, the fix/workaround is to pass -H 'Content-type:application/json' - JR-1991/pyDataverse@0fcfcd3

@Jeija are you testing with https://demo.dataverse.org ? Are you using your own server?

I opened this issue in the "operational work" issue tracker because I thought the demo server needed to be fixed but perhaps a code/doc change is needed.

[Jeija]

For clarification, I encountered the issue reported in gdcc/pyDataverse#143 while trying to upload something to the Dataverse installation of the University of Stuttgart at https://darus.uni-stuttgart.de/.

[scolapasta]

OK, after some discussion, we're happy with updating the guides to include -H 'Content-type:application/json' in the API instructions / curl examples. We'll also want to include a release note that says something to the effect of "some of the example command lines in previous versions of the guide omitted the content header… without the content header, the calls may fail"

Based on this, will also transfer to the other repo.