New expanded geospatial block by lubitchv · Pull Request #11507 · IQSS/dataverse

lubitchv · 2025-05-21T19:55:01Z

What this PR does / why we need it: Expanded geospatial block is proposed

Which issue(s) this PR closes:

Closes Feature Request/Idea: ISO 19115 versus Dataverse Geospatial Metadata Block (Proposed Changes) #10398

Special notes for your reviewer:
It is a new experimental geospatial block that extends the old one.

ofahimIQSS · 2025-05-27T15:14:01Z

@lubitchv Could you also add the new tsv file and the .properties file.
Does it also require an SQL migration script? Thanks!

lubitchv · 2025-05-27T20:41:54Z

@ofahimIQSS The new geospatial block is an extended version of an old geospatial.tsv. I can upload the new block as geospatial.tsv which will replace the old one or I can add it as a geospatial_new.tsv. In this case it may fail some tests since it is the extension of an old one and hence may conflict with it. What is better way to do? I think the end goal is update the existing block but I am not sure at this stage what should be done.

The new block does not need SQL migration script, since it is extension of old. I wrote an instruction in docs (in this PR) how to update existing installation with this extension of geospatial block.

ofahimIQSS · 2025-05-27T20:54:25Z

Thanks, @lubitchv ! We can go ahead and use geospatial.tsv as-is—no need to rename it. We'll test it on our end and roll it out accordingly.

lubitchv · 2025-05-28T22:01:25Z

I updated geospatial.tsv to a new expanded one. I also updated geospatial property file with new fields and also solr schema.xml I see that integration test is failing. Can I have a name of the test that fails?

pdurbin

Some feedback based on the latest commits. This PR is looking good!

pdurbin · 2025-06-02T14:34:49Z

doc/sphinx-guides/source/user/appendix.rst

+.. code-block:: javascript
+
+  curl http://localhost:8080/api/admin/datasetfield/load -H "Content-type: text/tab-separated-values" -X POST --upload-file geospatial_new.tsv
+  curl "http://localhost:8080/api/admin/index/solr/schema" > new.xml
+  ./dataverse/conf/solr/update-fields.sh /usr/local/solr/solr-9.8.0/server/solr/collection1/conf/schema.xml new.xml
+  curl "http://localhost:8983/solr/admin/cores?action=RELOAD&core=collection1"


@lubitchv these upgrade instructions are great, thanks. Let's move them to a release note snippet.

I created the release snippet for upgrade.

pdurbin · 2025-06-02T14:37:06Z

src/main/java/propertyFiles/geospatial.properties

+controlledvocabulary.spatialResolutionType.distance=distance
+controlledvocabulary.spatialResolutionType.vertical=vertical
+controlledvocabulary.spatialResolutionType.angulardistance=angularDistance
+controlledvocabulary.spatialResolutionType.levelofdetail=levelOfDetail


Here at the bottom let's track failing tests. This is what I'm seeing right now:

edu.harvard.iq.dataverse.api.DatasetFieldsIT.testListAllFacetableDatasetFields
edu.harvard.iq.dataverse.api.DatasetTypesIT.testUpdateDatasetTypeLinksWithMetadataBlocks
edu.harvard.iq.dataverse.api.DataversesIT.testListMetadataBlocks
edu.harvard.iq.dataverse.api.MetadataBlocksIT.testListMetadataBlocks

From a quick look they all have to do with the number of fields expected, like this:

JSON path data.size() doesn't match. Expected: <64> Actual: <71>

I fixed the integration tests.

pdurbin · 2025-06-02T14:38:57Z

doc/sphinx-guides/source/user/appendix.rst

 - Computational Workflow Metadata (`see .tsv <https://github.com/IQSS/dataverse/blob/master/scripts/api/data/metadatablocks/computational_workflow.tsv>`__): adapted from `Bioschemas Computational Workflow Profile, version 1.0 <https://bioschemas.org/profiles/ComputationalWorkflow/1.0-RELEASE>`__ and `Codemeta <https://codemeta.github.io/terms/>`__.
 - Archival Metadata (`see .tsv <https://github.com/IQSS/dataverse/blob/master/scripts/api/data/metadatablocks/archival.tsv>`__): Enables repositories to register metadata relating to the potential archiving of the dataset at a depositor archive, whether that be your own institutional archive or an external archive, i.e. a historical archive.
-
+- `New updated Geospatial Metadata block <https://docs.google.com/spreadsheets/d/1GMVIaQB6EqauNDqrPq09BBHd_HzUSD57oliTAoh3Ikw>`__: adapted for ISO 19115-3 format. It substitutes and expands existing geospatial.tsv metadata block. To use it, replace existing geospatial.tsv with the new one. To upload the block to the existing installation:


Let's check with @jggautier and others but I believe we we should overwrite "Geospatial Metadata" above rather than having a separate entry. In a release note snippet we can explain that the geospatial block has been updated.

On a related note, rather than "New expanded geospatial block" should the title of this PR be "Updated geospatial block" or "updated" or "improved" or "extended" or whatever. It's not really new, is it? It's bigger and better. 😄

pdurbin · 2025-06-04T21:07:07Z

scripts/api/data/metadatablocks/geospatial.tsv

-	eastLongitude	Easternmost (Right) Longitude	Easternmost coordinate delimiting the geographic extent of the Dataset. A valid range of values,  expressed in decimal degrees, is -180.0 <= East Bounding Longitude Value <= 180.0.		text	8		FALSE	FALSE	FALSE	FALSE	FALSE	FALSE	geographicBoundingBox	geospatial
-	northLatitude	Northernmost (Top) Latitude	Northernmost coordinate delimiting the geographic extent of the Dataset. A valid range of values,  expressed in decimal degrees, is -90.0 <= North Bounding Latitude Value <= 90.0.		text	9		FALSE	FALSE	FALSE	FALSE	FALSE	FALSE	geographicBoundingBox	geospatial
-	southLatitude	Southernmost (Bottom) Latitude	Southernmost coordinate delimiting the geographic extent of the Dataset. A valid range of values,  expressed in decimal degrees, is -90.0 <= South Bounding Latitude Value <= 90.0.		text	10		FALSE	FALSE	FALSE	FALSE	FALSE	FALSE	geographicBoundingBox	geospatial
+	geographicCoverage	Geographic Coverage	Information on the geographic coverage of the data. Includes the total geographic scope of the data.		none	0		FALSE	FALSE	TRUE	FALSE	TRUE	FALSE		geospatial


Someone at standup noticed that you've changed displayoncreate for a lot of field from false to true:

pdurbin@beamish dataverse % git checkout develop Switched to branch 'develop' Your branch is up to date with 'origin/develop'. pdurbin@beamish dataverse % cat scripts/api/data/metadatablocks/geospatial.tsv | head | cut -f2,13 name geospatial name displayoncreate geographicCoverage FALSE country FALSE state FALSE city FALSE otherGeographicCoverage FALSE geographicUnit FALSE geographicBoundingBox FALSE pdurbin@beamish dataverse % git checkout 10398-geospatial-block Switched to branch '10398-geospatial-block' Your branch is up to date with 'lubitchv/10398-geospatial-block'. pdurbin@beamish dataverse % cat scripts/api/data/metadatablocks/geospatial.tsv | head | cut -f2,13 name geospatial name displayoncreate geographicCoverage TRUE country TRUE state TRUE city TRUE otherGeographicCoverage TRUE geographicUnit TRUE geographicBoundingBox TRUE

Can you please switch them back to false?

On a related note, there's a new API added in #11001 for 6.6, that allows you to change displayoncreate: https://guides.dataverse.org/en/6.6/api/native-api.html#set-displayoncreate-for-a-dataset-field . So maybe you can use that to set fields to true for your installation.

I replaced back displayoncreate to all the fields to FALSE

jggautier · 2025-06-09T12:29:30Z

Hi all! Just got back from vacation and noticed I was asked if this metadata block should be described in the "Experimental Metadata" section or if its description should replace what's in the "Supported Metadata" section about geospatial metadata.

@lubitchv could you write why you added a description in the "Experimental Metadata" section? From the discussion in this PR so far I couldn't get a sense of why.

And where ever the metadata block is described on this Appendix page, should it include information about how to upload the metadata block? I ask because this would be the first time we include that sort of information here, when I think it's usually in the release notes and in the Metadata Customization section of the Admin Guide.

jggautier · 2025-06-10T17:45:05Z

Just an update from my comments and questions yesterday after I got to chat briefly with @lubitchv this morning at the Dataverse Community Meeting at UNC. We agreed to try to find time to discuss more with more folks, including @amberleahey.

I should also say that I just noticed that "It is a new experimental geospatial block that extends the old one" is what's in the "Special notes for your reviewer" section of this PR's first comment, so it sort of makes sense to me why you'd want to describe it in the Appendix's "Experimental Metadata" section. And I think it would help to discuss with @pdurbin why he's encouraging you to instead replace what's in the Appendix's "Supported Metadata" section.

Hopefully we'll find time to chat this week during the conference.

pdurbin · 2025-06-17T18:14:21Z

Absolutely! Active is better! And bringing in @emily-katz and @Saixel from CAFE is an excellent idea. 💡 I'm happy to join a meeting if that helps.

This PR changes the actual, real, production geospatial metadata block so once it's merged it is very much non-experimental.

Perhaps we could use the word "preview" instead? 🤔 Whatever gets across the idea that we'd like people to try the updated metadata block before we merge. 😄

jp-tosca · 2025-06-18T15:09:25Z

Moved to hold

pdurbin · 2025-07-29T20:22:19Z

Here's the already-merged PR that updated the appendix to encourage people to try the version of the geospatial block in this PR (#11507):

In guides, suggest trying an update to the geospatial metadata block #11640

At some point @amberleahey and @lubitchv will have enough feedback that they'll make modifications or ask to move this PR forward as-is.

Update from my synced develop

coveralls · 2025-09-04T16:01:32Z

coverage: 24.33% (+0.2%) from 24.168%
when pulling a0eb3d5 on lubitchv:10398-geospatial-block
into d2b6a46 on IQSS:develop.

lubitchv · 2025-10-24T15:24:35Z

Can you let me know what integration test is failing?

qqmyers · 2025-10-24T15:45:21Z

It's a build issue - I merged develop into your branch to see if that helps.

amberleahey · 2025-12-12T17:52:39Z

Made a comment on #1592
DV-ISO Metadata Crosswalks.xlsx
ForReview_DataverseNewGeoMetadataBlock_12-2025.xlsx
Stay tuned...

qqmyers · 2026-02-03T15:28:28Z

scripts/api/data/metadatablocks/geospatial.tsv

-	country	Zimbabwe		247											
-	country	Åland Islands		248											
+#metadataBlock	name	dataverseAlias	displayName												
+	geospatial		Geospatial Metadata												


Have you considered a blockUri and/or termUri entries to make the association with 19115 clearer? As an XML standard they may not have a good mapping to URIs/RDF, but it might still make sense to add URI mapping even if they are of the form https://dataverse.org/vocab/iso19115/* (especially if not all fields in the block map to 19115)

new geospatial block

7e9ef14

pdurbin added this to IQSS Dataverse Project May 21, 2025

pdurbin moved this to Ready for Triage in IQSS Dataverse Project May 21, 2025

remove block

e17a01b

ofahimIQSS assigned lubitchv May 27, 2025

lubitchv added 7 commits May 28, 2025 10:54

expanded geospatial block

f85e079

geospatial properties

81e7459

geospatial properties fix

9b343cd

geospatial properties fix

e724514

geospatial.tsv

ab6d9a1

updated schema

93f5551

fix geospatial.tsv

96be722

pdurbin reviewed Jun 2, 2025

View reviewed changes

ofahimIQSS added the Size: 10 A percentage of a sprint. 7 hours. label Jun 3, 2025

ofahimIQSS assigned pdurbin Jun 3, 2025

ofahimIQSS moved this from Ready for Triage to Ready for Review ⏩ in IQSS Dataverse Project Jun 3, 2025

ofahimIQSS moved this from Ready for Review ⏩ to In Review 🔎 in IQSS Dataverse Project Jun 3, 2025

cmbz added FY25 Sprint 24 FY25 Sprint 24 (2025-05-21 - 2025-06-04) FY25 Sprint 25 FY25 Sprint 25 (2025-06-04 - 2025-06-18) labels Jun 4, 2025

pdurbin reviewed Jun 4, 2025

View reviewed changes

lubitchv added 4 commits June 5, 2025 16:35

geo false on creation

fd7072c

geo fix

9640193

int test naumber of fields

ea4327f

upgrade snippet

3d4c125

pdurbin mentioned this pull request Jun 17, 2025

Update the codemeta metadata block to add some more structure for machine actionability #11087

Open

jp-tosca moved this from In Review 🔎 to On Hold ⌛ in IQSS Dataverse Project Jun 18, 2025

qqmyers mentioned this pull request Jun 20, 2025

Feature Request: add bounding boxes to all metadata formats based on schema.org #11582

Open

pdurbin added a commit that referenced this pull request Jul 16, 2025

link to #11507 from geospatial bullet #11640

d55f04d

pdurbin assigned amberleahey and unassigned pdurbin Jul 29, 2025

lubitchv added 3 commits September 4, 2025 11:42

Merge branch '10398-geospatial-block' into develop_geo

8b11565

Merge pull request #73 from lubitchv/develop_geo

b102ebd

Update from my synced develop

Remove empy line

41ba53b

lubitchv and others added 5 commits September 4, 2025 17:22

Updated geoblock

11fb9f6

integration tests

d8920e1

Merge branch 'IQSS:develop' into 10398-geospatial-block

f8d0614

change to reolution

2302973

update geo block

92af478

Merge branch 'develop' into 10398-geospatial-block

0f34ee0

DS-INRAE mentioned this pull request Oct 30, 2025

Integrate Geographic Unit Field into Geographic Coverage Metadata Block #10502

Open

Merge branch 'IQSS:develop' into 10398-geospatial-block

1643e76

amberleahey mentioned this pull request Dec 12, 2025

Support for Importing and Exporting DCAT Metadata #1592

Open

amberleahey mentioned this pull request Dec 15, 2025

Feature Request/Idea: ISO 19115 versus Dataverse Geospatial Metadata Block (Proposed Changes) #10398

Open

lubitchv and others added 3 commits January 22, 2026 11:49

geospatial.tsv

a0eb3d5

Merge branch 'IQSS:develop' into 10398-geospatial-block

b79d38b

removed quotes

daae81a

qqmyers reviewed Feb 3, 2026

View reviewed changes

Conversation

lubitchv commented May 21, 2025

Uh oh!

ofahimIQSS commented May 27, 2025

Uh oh!

lubitchv commented May 27, 2025

Uh oh!

ofahimIQSS commented May 27, 2025

Uh oh!

lubitchv commented May 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pdurbin left a comment

Choose a reason for hiding this comment

Uh oh!

pdurbin Jun 2, 2025

Choose a reason for hiding this comment

Uh oh!

lubitchv Jun 7, 2025

Choose a reason for hiding this comment

Uh oh!

pdurbin Jun 2, 2025

Choose a reason for hiding this comment

Uh oh!

lubitchv Jun 7, 2025

Choose a reason for hiding this comment

Uh oh!

pdurbin Jun 2, 2025

Choose a reason for hiding this comment

Uh oh!

pdurbin Jun 4, 2025

Choose a reason for hiding this comment

Uh oh!

lubitchv Jun 5, 2025

Choose a reason for hiding this comment

Uh oh!

jggautier commented Jun 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jggautier commented Jun 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pdurbin commented Jun 17, 2025

Uh oh!

jp-tosca commented Jun 18, 2025

Uh oh!

pdurbin commented Jul 29, 2025

Uh oh!

coveralls commented Sep 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lubitchv commented Oct 24, 2025

Uh oh!

qqmyers commented Oct 24, 2025

Uh oh!

amberleahey commented Dec 12, 2025

Uh oh!

qqmyers Feb 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

lubitchv commented May 28, 2025 •

edited

Loading

jggautier commented Jun 9, 2025 •

edited

Loading

jggautier commented Jun 10, 2025 •

edited

Loading

coveralls commented Sep 4, 2025 •

edited

Loading

qqmyers Feb 3, 2026 •

edited

Loading