Dataset publishing prevented or slowed by errors related to possible validation of metadata entered in Geospatial Bounding Box fields

[jggautier]

What steps does it take to reproduce the issue?

• When does this issue occur?
  When using a web browser to try to create or edit a dataset version that includes "invalid" data in the Bounding Box fields in the Geospatial metadata block.

  And when trying to create a dataset version using the Native API's "Create a dataset in a Dataverse collection" and "Import a Dataset into a Dataverse Collection" endpoints while the JSON file that's passed in the API call includes "invalid" data in Geospatial Bounding Box fields.

• Which page(s) does it occurs on?
  The dataset create and edit pages.

• What happens?
  When using the API endpoints, the dataset version can't be created or updated. The endpoints return two error messages, one error message when there are non-numerical values in any of the Geospatial Bounding Box fields, and another error message when the number in the North Latitude field is less than the number in the South Latitude field.

  When using a web browser, the dataset can be saved, eventually. After clicking Save Changes I get the error "Error – The metadata could not be updated. If you believe this is an error, please contact Demo Dataverse Support for assistance." but I can see that the metadata was updated.

  The Publish button doesn't work until I refresh the page, which removes the error message. Then I can publish the dataset.

  And the dataset URL includes "&version=DRAFT" while the page shows the Info banner that reads that a draft version wasn't found. This might be related to an already documented bug.

• To whom does it occur (all users, curators, superusers)?
  Superusers, curators, depositors

• What did you expect to happen?
  Has validation been added to the Geospatial Bounding Box fields, maybe as part of the Geospatial Search added in v5.13 (Geospatial search #8239)? If it has, the error message in the GUI should make clear what's preventing the dataset version from being saved. The API endpoints return more specific error messages, but I'm not sure if they are clear to others.

Which version of Dataverse are you using?
v5.13

[jggautier]

In Dataverse installations running v5.13, this "invalid" bounding box metadata will prevent datasets from being updated.

An example in the Harvard Dataverse Repository is the dataset at https://doi.org/10.7910/DVN/1HTKPJ.

As other installations update their Dataverse software to v5.13, their datasets with this "invalid" metadata also won't be able to be updated.

An example in HeiDATA's installation is the dataset at https://doi.org/10.11588/data/LSG8TN. An example in Open Forest Data is the dataset at https://doi.org/10.48370/OFD/008Y03. For many of the datasets in that repository, commas are used instead of periods.

People who try to use their browsers to update datasets like these will see an error message with no information about what metadata could not be updated:

[pdurbin]

maybe as part of the Geospatial Search added in v5.13

Yes, I suspect this is a Solr error and due to the addition of geospatial search.

What should we do with bounding box data? Save it anyway like we always have, and not index it into Solr?

Or should the user get a validation error "lat/long wrong blah blah" that prevents them from saving, similar to an invalid URL or email address?

[mreekie]

grooming

• Looking at the NetCDF deliverables.
• This looks like it's a related issue that is related to active work, but it is not in the netcdf backlog Queue
• Not going to add it to the queue, but tagging it.

[jggautier]

#9547 is related.

[meghangoodchild]

Just wanted to add that we experienced this issue at Borealis due to invalid metadata in the geographic bounding box fields, which prevented proper indexing of the dataset.
It would be helpful if there was a validation error that points to the issue. It took us awhile to figure it out, and we also had trouble republishing, even after removing the invalid metadata.
Thanks!

[pdurbin]

@meghangoodchild yikes. Thanks for the heads up. I'm not surprised that installations are starting to see this problem.

Like @jggautier noted, the following issue is related, and we could probably collapse it into one effort:

• Add validation for entered coordinate values in Geospatial metadata (bounding boxes, etc) #9547

Also, I was poking at #9593 the other day and ran into an indexing problem: #9595 (comment)

It would be nice if any fix we put it ultimately sends only the numerical values to Solr (after validation, of course), to unblock that issue and pull request.

[cmbz]

2023/10/02: Depending upon sizing, if small add to 6.1, otherwise we can defer. There is a workaround, if necessary.