From 28ff8ed0dfe3a975d273f5c7cf8430fb99434498 Mon Sep 17 00:00:00 2001 From: Dorothea Iglezakis Date: Wed, 11 Dec 2024 15:45:34 +0100 Subject: [PATCH 1/6] added subfields and controlled vocabularies for memoryRequirements and storageRequirements, added new subfield for softwareRequirements and softwareSuggestions to distinguish between InfoUrl (documentation page) and URL (DownloadURL) for dependencies, adjusted termURI of contIntegration to codemeta v3.0 in codemeta.tsv --- scripts/api/data/metadatablocks/codemeta.tsv | 45 ++++++++++++++------ 1 file changed, 32 insertions(+), 13 deletions(-) diff --git a/scripts/api/data/metadatablocks/codemeta.tsv b/scripts/api/data/metadatablocks/codemeta.tsv index a5c50368b75..e6f31a2822d 100644 --- a/scripts/api/data/metadatablocks/codemeta.tsv +++ b/scripts/api/data/metadatablocks/codemeta.tsv @@ -1,5 +1,5 @@ #metadataBlock name dataverseAlias displayName blockURI - codeMeta20 Software Metadata (CodeMeta v2.0) https://codemeta.github.io/terms/ + codeMeta20 Software Metadata (CodeMeta 2.0) https://codemeta.github.io/terms/ #datasetField name title description watermark fieldType displayOrder displayFormat advancedSearchField allowControlledVocabulary allowmultiples facetable displayoncreate required parent metadatablock_id termURI codeVersion Software Version Version of the software instance, usually following some convention like SemVer etc. e.g. 0.2.1 or 1.3 or 2021.1 etc text 0 #VALUE TRUE FALSE FALSE TRUE TRUE FALSE codeMeta20 https://schema.org/softwareVersion developmentStatus Development Status Description of development status, e.g. work in progress (wip), active, etc. See repostatus.org for more information. text 1 #VALUE TRUE TRUE FALSE TRUE FALSE FALSE codeMeta20 https://www.repostatus.org @@ -12,20 +12,27 @@ targetProduct Target Product Target Operating System / Product to which the code applies. If applies to several versions, just the product name can be used. text 8 #VALUE TRUE FALSE TRUE TRUE FALSE FALSE codeMeta20 https://schema.org/targetProduct buildInstructions Build Instructions Link to installation instructions/documentation e.g. https://github.com/user/project/blob/main/BUILD.md url 9 #VALUE FALSE FALSE TRUE FALSE FALSE FALSE codeMeta20 https://codemeta.github.io/terms/buildInstructions softwareRequirementsItem Software Requirements Required software dependencies none 10 FALSE FALSE TRUE FALSE TRUE FALSE codeMeta20 - softwareRequirements Name & Version Name and version of the required software/library dependency e.g. Pandas 1.4.3 text 0 #VALUE TRUE FALSE FALSE FALSE TRUE FALSE softwareRequirementsItem codeMeta20 https://schema.org/softwareRequirements + softwareRequirements Name & Version Name and version of the required software/library dependency e.g. Pandas 1.4.3 text 0 #VALUE TRUE FALSE FALSE FALSE TRUE TRUE softwareRequirementsItem codeMeta20 https://schema.org/softwareRequirements softwareRequirementsInfoUrl Info URL Link to required software/library homepage or documentation (ideally also versioned) e.g. https://pandas.pydata.org/pandas-docs/version/1.4.3 url 1 #VALUE FALSE FALSE FALSE FALSE TRUE FALSE softwareRequirementsItem codeMeta20 https://dataverse.org/schema/codeMeta20/softwareRequirementsInfoUrl + softwareRequirementsUrl Download URL Link to required software/library https://... url 2 #VALUE FALSE FALSE FALSE FALSE FALSE FALSE softwareRequirementsItem codeMeta20 softwareSuggestionsItem Software Suggestions Optional dependencies, e.g. for optional features, code development, etc. none 11 FALSE FALSE TRUE FALSE FALSE FALSE codeMeta20 - softwareSuggestions Name & Version Name and version of the optional software/library dependency e.g. Sphinx 5.0.2 text 0 #VALUE TRUE FALSE FALSE TRUE FALSE FALSE softwareSuggestionsItem codeMeta20 https://codemeta.github.io/terms/softwareSuggestions - softwareSuggestionsInfoUrl Info URL Link to optional software/library homepage or documentation (ideally also versioned) e.g. https://www.sphinx-doc.org url 1 #VALUE FALSE FALSE FALSE FALSE FALSE FALSE softwareSuggestionsItem codeMeta20 https://dataverse.org/schema/codeMeta20/softwareSuggestionsInfoUrl - memoryRequirements Memory Requirements Minimum memory requirements. text 12 #VALUE TRUE FALSE FALSE FALSE FALSE FALSE codeMeta20 https://schema.org/memoryRequirements - processorRequirements Processor Requirements Processor architecture or other CPU requirements to run the application (e.g. IA64). text 13 #VALUE TRUE FALSE TRUE FALSE FALSE FALSE codeMeta20 https://schema.org/processorRequirements - storageRequirements Storage Requirements Minimum storage requirements (e.g. free space required). text 14 #VALUE TRUE FALSE FALSE FALSE FALSE FALSE codeMeta20 https://schema.org/storageRequirements - permissions Permissions Permission(s) required to run the code (for example, a mobile app may require full internet access or may run only on wifi). text 15 #VALUE TRUE FALSE TRUE FALSE FALSE FALSE codeMeta20 https://schema.org/permissions - softwareHelp Software Help/Documentation Link to help texts or documentation e.g. https://user.github.io/project/docs url 16 #VALUE FALSE FALSE TRUE FALSE TRUE FALSE codeMeta20 https://schema.org/softwareHelp - readme Readme Link to the README of the project e.g. https://github.com/user/project/blob/main/README.md url 17 #VALUE FALSE FALSE FALSE FALSE FALSE FALSE codeMeta20 https://codemeta.github.io/terms/readme - releaseNotes Release Notes Link to release notes e.g. https://github.com/user/project/blob/main/docs/release-0.1.md url 18 #VALUE FALSE FALSE FALSE FALSE FALSE FALSE codeMeta20 https://schema.org/releaseNotes - contIntegration Continuous Integration Link to continuous integration service e.g. https://github.com/user/project/actions url 19 #VALUE FALSE FALSE TRUE FALSE FALSE FALSE codeMeta20 https://codemeta.github.io/terms/contIntegration - issueTracker Issue Tracker Link to software bug reporting or issue tracking system e.g. https://github.com/user/project/issues url 20 #VALUE FALSE FALSE FALSE FALSE FALSE FALSE codeMeta20 https://codemeta.github.io/terms/issueTracker + softwareSuggestions Name & Version Name and version of the optional software/library dependency e.g. Sphinx 5.0.2 text 0 #VALUE TRUE FALSE FALSE TRUE FALSE TRUE softwareSuggestionsItem codeMeta20 + softwareSuggestionsInfoUrl Info URL Link to optional software/library homepage or documentation (ideally also versioned) e.g. https://www.sphinx-doc.org url 1 #VALUE FALSE FALSE FALSE FALSE FALSE FALSE softwareSuggestionsItem codeMeta20 + softwareSuggestionsUrl Download URL Link to optional software/library https://... url 2 #VALUE FALSE FALSE FALSE FALSE FALSE FALSE softwareSuggestionsItem codeMeta20 + memoryRequirements Memory Requirements Minimum memory requirements text 12 TRUE FALSE TRUE FALSE FALSE FALSE codeMeta20 https://schema.org/memoryRequirements + memoryRequirementsSize Memory Size Minimum memory requirements size int 13 #VALUE TRUE FALSE FALSE FALSE FALSE FALSE memoryRequirements codeMeta20 + memoryRequirementsUnit Memory Unit Memory Unit (KB, MB, GB, TB) text 14 #VALUE TRUE TRUE FALSE FALSE FALSE FALSE memoryRequirements codeMeta20 + memoryRequirementsType Memory Type Type of memory (GPU or RAM) text 15 (#VALUE) TRUE TRUE FALSE FALSE FALSE FALSE memoryRequirements codeMeta20 + processorRequirements Processor Requirements Processor architecture or other CPU requirements to run the application (e.g. IA64). text 16 #VALUE TRUE FALSE TRUE FALSE FALSE FALSE codeMeta20 https://schema.org/processorRequirements + storageRequirements Storage Requirements Minimum storage requirements (e.g. free space required). text 17 TRUE FALSE FALSE FALSE FALSE FALSE codeMeta20 https://schema.org/storageRequirements + storageRequirementsSize Storage Size Minimum storage requirements size text 18 #VALUE TRUE FALSE FALSE FALSE FALSE FALSE storageRequirements codeMeta20 + storageRequirementsUnit Storage Unit Storage Unit (MB, GB, TB) text 19 #VALUE TRUE TRUE FALSE FALSE FALSE FALSE storageRequirements codeMeta20 + permissions Permissions Permission(s) required to run the code (for example, a mobile app may require full internet access or may run only on wifi). text 20 #VALUE TRUE FALSE TRUE FALSE FALSE FALSE codeMeta20 https://schema.org/permissions + softwareHelp Software Help/Documentation Link to help texts or documentation e.g. https://user.github.io/project/docs url 21 #VALUE FALSE FALSE TRUE FALSE TRUE FALSE codeMeta20 https://schema.org/softwareHelp + readme Readme Link to the README of the project e.g. https://github.com/user/project/blob/main/README.md url 22 #VALUE FALSE FALSE FALSE FALSE FALSE FALSE codeMeta20 https://codemeta.github.io/terms/readme + releaseNotes Release Notes Link to release notes e.g. https://github.com/user/project/blob/main/docs/release-0.1.md url 23 #VALUE FALSE FALSE FALSE FALSE FALSE FALSE codeMeta20 https://schema.org/releaseNotes + contIntegration Continuous Integration Link to continuous integration service e.g. https://github.com/user/project/actions url 24 #VALUE FALSE FALSE TRUE FALSE FALSE FALSE codeMeta20 https://codemeta.github.io/terms/continuousIntegration + issueTracker Issue Tracker Link to software bug reporting or issue tracking system e.g. https://github.com/user/project/issues url 25 #VALUE FALSE FALSE FALSE FALSE FALSE FALSE codeMeta20 https://codemeta.github.io/terms/issueTracker #controlledVocabulary DatasetField Value identifier displayOrder developmentStatus Concept concept 0 developmentStatus WIP wip 1 @@ -35,3 +42,15 @@ developmentStatus Moved moved 5 developmentStatus Suspended suspended 6 developmentStatus Abandoned abandoned 7 + memoryRequirementsType RAM ram 1 + memoryRequirementsType GPU gpu 2 + memoryRequirementsType NPU npu 3 + memoryRequirementsUnit KB kb 1 + memoryRequirementsUnit MB mb 2 + memoryRequirementsUnit GB gb 3 + memoryRequirementsUnit TB tb 4 + storageRequirementsUnit KB kb 1 + storageRequirementsUnit MB mb 2 + storageRequirementsUnit GB gb 3 + storageRequirementsUnit TB tb 4 + storageRequirementsUnit PB pb 5 From 23c8f74e3a48f9f721619da5a0a0f59ed6234ba1 Mon Sep 17 00:00:00 2001 From: Dorothea Iglezakis Date: Fri, 7 Feb 2025 15:17:56 +0100 Subject: [PATCH 2/6] changed display name of codemeta metadata block to version 3.0 --- scripts/api/data/metadatablocks/codemeta.tsv | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/api/data/metadatablocks/codemeta.tsv b/scripts/api/data/metadatablocks/codemeta.tsv index e6f31a2822d..b50408bc28b 100644 --- a/scripts/api/data/metadatablocks/codemeta.tsv +++ b/scripts/api/data/metadatablocks/codemeta.tsv @@ -1,5 +1,5 @@ #metadataBlock name dataverseAlias displayName blockURI - codeMeta20 Software Metadata (CodeMeta 2.0) https://codemeta.github.io/terms/ + codeMeta20 Software Metadata (CodeMeta v3.0) https://codemeta.github.io/terms/ #datasetField name title description watermark fieldType displayOrder displayFormat advancedSearchField allowControlledVocabulary allowmultiples facetable displayoncreate required parent metadatablock_id termURI codeVersion Software Version Version of the software instance, usually following some convention like SemVer etc. e.g. 0.2.1 or 1.3 or 2021.1 etc text 0 #VALUE TRUE FALSE FALSE TRUE TRUE FALSE codeMeta20 https://schema.org/softwareVersion developmentStatus Development Status Description of development status, e.g. work in progress (wip), active, etc. See repostatus.org for more information. text 1 #VALUE TRUE TRUE FALSE TRUE FALSE FALSE codeMeta20 https://www.repostatus.org From 00fee13a78159a1c620baf03fb998ff3cdba0003 Mon Sep 17 00:00:00 2001 From: Dorothea Iglezakis Date: Fri, 7 Feb 2025 15:18:21 +0100 Subject: [PATCH 3/6] added release note snippet --- doc/release-notes/11087-codemeta-block-improvement.md | 6 ++++++ 1 file changed, 6 insertions(+) create mode 100644 doc/release-notes/11087-codemeta-block-improvement.md diff --git a/doc/release-notes/11087-codemeta-block-improvement.md b/doc/release-notes/11087-codemeta-block-improvement.md new file mode 100644 index 00000000000..504447ff673 --- /dev/null +++ b/doc/release-notes/11087-codemeta-block-improvement.md @@ -0,0 +1,6 @@ +The experimental codemeta metadatablock is improved by: +* Adding subfields for size, unit and type to memoryRequirements and subfields for size and unit for storage to storageRequirements of software to improve the machine actionability of these metadata fields and enable external tools like Jupyter labs to run the software in an appropriate environmen +* adding a new subfield InfoUrl to softwareSuggestions and softwareRequirements to distinguish between the download url of a dependency (URL) and an information page of a dependency (InfoUrl) +* adjusting the termURI of the contIntegration metadata field to the changes with CodeMeta v.3.0 + +Please note, that existing metadata contents of the fields memoryRequirements and storageRequirements have to be manually migrated to the new subfields. \ No newline at end of file From c0424021ab3c777733dbb02e08649f912da5e0f1 Mon Sep 17 00:00:00 2001 From: Dorothea Iglezakis Date: Fri, 7 Feb 2025 15:21:45 +0100 Subject: [PATCH 4/6] updated appendix information to codemeta version 3 --- doc/sphinx-guides/source/user/appendix.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/sphinx-guides/source/user/appendix.rst b/doc/sphinx-guides/source/user/appendix.rst index df9b6704209..9ec125e2684 100755 --- a/doc/sphinx-guides/source/user/appendix.rst +++ b/doc/sphinx-guides/source/user/appendix.rst @@ -36,7 +36,7 @@ Experimental Metadata Unlike supported metadata, experimental metadata is not enabled by default in a new Dataverse installation. Feedback via any `channel `_ is welcome! -- `CodeMeta Software Metadata `__: based on the `CodeMeta Software Metadata Schema, version 2.0 `__ (`see .tsv version `__) +- `CodeMeta Software Metadata `__: based on the `CodeMeta Software Metadata Schema, version 3.0 `__ (`see .tsv version `__) - Computational Workflow Metadata (`see .tsv `__): adapted from `Bioschemas Computational Workflow Profile, version 1.0 `__ and `Codemeta `__. Please note: these custom metadata schemas are not included in the Solr schema for indexing by default, you will need From b2f0a0fb009933aa21d5709c3a338117a18c5a2f Mon Sep 17 00:00:00 2001 From: Dorothea Iglezakis Date: Mon, 10 Feb 2025 19:40:32 +0100 Subject: [PATCH 5/6] added adjusted properties file for codeMeta20 metadatablock --- .../java/propertyFiles/codeMeta20.properties | 39 +++++++++++++++++-- 1 file changed, 36 insertions(+), 3 deletions(-) diff --git a/src/main/java/propertyFiles/codeMeta20.properties b/src/main/java/propertyFiles/codeMeta20.properties index 4f3eb087aa4..5b429df0fb3 100644 --- a/src/main/java/propertyFiles/codeMeta20.properties +++ b/src/main/java/propertyFiles/codeMeta20.properties @@ -1,12 +1,12 @@ metadatablock.name=codeMeta20 -metadatablock.displayName=Software Metadata (CodeMeta v2.0) +metadatablock.displayName=Software Metadata (CodeMeta v3.0) metadatablock.displayFacet=Software datasetfieldtype.codeVersion.title=Software Version datasetfieldtype.codeVersion.description=Version of the software instance, usually following some convention like SemVer etc. datasetfieldtype.codeVersion.watermark=e.g. 0.2.1 or 1.3 or 2021.1 etc datasetfieldtype.developmentStatus.title=Development Status datasetfieldtype.developmentStatus.description=Description of development status, e.g. work in progress (wip), active, etc. See repostatus.org for more information. -datasetfieldtype.developmentStatus.watermark= Development Status +datasetfieldtype.developmentStatus.watermark= datasetfieldtype.codeRepository.title=Code Repository datasetfieldtype.codeRepository.description=Link to the repository where the un-compiled, human-readable code and related code is located (SVN, GitHub, CodePlex, institutional GitLab instance, Gitea, etc.). datasetfieldtype.codeRepository.watermark=e.g. https://github.com/user/project @@ -40,6 +40,9 @@ datasetfieldtype.softwareRequirements.watermark=e.g. Pandas 1.4.3 datasetfieldtype.softwareRequirementsInfoUrl.title=Info URL datasetfieldtype.softwareRequirementsInfoUrl.description=Link to required software/library homepage or documentation (ideally also versioned) datasetfieldtype.softwareRequirementsInfoUrl.watermark=e.g. https://pandas.pydata.org/pandas-docs/version/1.4.3 +datasetfieldtype.softwareRequirementsUrl.title=Download URL +datasetfieldtype.softwareRequirementsUrl.description=Link to required software/library +datasetfieldtype.softwareRequirementsUrl.watermark=https://... datasetfieldtype.softwareSuggestionsItem.title=Software Suggestions datasetfieldtype.softwareSuggestionsItem.description=Optional dependencies, e.g. for optional features, code development, etc. datasetfieldtype.softwareSuggestionsItem.watermark= @@ -49,15 +52,33 @@ datasetfieldtype.softwareSuggestions.watermark=e.g. Sphinx 5.0.2 datasetfieldtype.softwareSuggestionsInfoUrl.title=Info URL datasetfieldtype.softwareSuggestionsInfoUrl.description=Link to optional software/library homepage or documentation (ideally also versioned) datasetfieldtype.softwareSuggestionsInfoUrl.watermark=e.g. https://www.sphinx-doc.org +datasetfieldtype.softwareSuggestionsUrl.title=Download URL +datasetfieldtype.softwareSuggestionsUrl.description=Link to optional software/library +datasetfieldtype.softwareSuggestionsUrl.watermark=https://... datasetfieldtype.memoryRequirements.title=Memory Requirements -datasetfieldtype.memoryRequirements.description=Minimum memory requirements. +datasetfieldtype.memoryRequirements.description=Minimum memory requirements datasetfieldtype.memoryRequirements.watermark= +datasetfieldtype.memoryRequirementsSize.title=Memory Size +datasetfieldtype.memoryRequirementsSize.description=Minimum memory requirements size +datasetfieldtype.memoryRequirementsSize.watermark= +datasetfieldtype.memoryRequirementsUnit.title=Memory Unit +datasetfieldtype.memoryRequirementsUnit.description=Memory Unit (KB, MB, GB, TB) +datasetfieldtype.memoryRequirementsUnit.watermark= +datasetfieldtype.memoryRequirementsType.title=Memory Type +datasetfieldtype.memoryRequirementsType.description=Type of memory (GPU or RAM) +datasetfieldtype.memoryRequirementsType.watermark= datasetfieldtype.processorRequirements.title=Processor Requirements datasetfieldtype.processorRequirements.description=Processor architecture or other CPU requirements to run the application (e.g. IA64). datasetfieldtype.processorRequirements.watermark= datasetfieldtype.storageRequirements.title=Storage Requirements datasetfieldtype.storageRequirements.description=Minimum storage requirements (e.g. free space required). datasetfieldtype.storageRequirements.watermark= +datasetfieldtype.storageRequirementsSize.title=Storage Size +datasetfieldtype.storageRequirementsSize.description=Minimum storage requirements size +datasetfieldtype.storageRequirementsSize.watermark= +datasetfieldtype.storageRequirementsUnit.title=Storage Unit +datasetfieldtype.storageRequirementsUnit.description=Storage Unit (MB, GB, TB) +datasetfieldtype.storageRequirementsUnit.watermark= datasetfieldtype.permissions.title=Permissions datasetfieldtype.permissions.description=Permission(s) required to run the code (for example, a mobile app may require full internet access or may run only on wifi). datasetfieldtype.permissions.watermark= @@ -84,3 +105,15 @@ controlledvocabulary.developmentStatus.unsupported=Unsupported controlledvocabulary.developmentStatus.moved=Moved controlledvocabulary.developmentStatus.suspended=Suspended controlledvocabulary.developmentStatus.abandoned=Abandoned +controlledvocabulary.memoryRequirementsType.ram=RAM +controlledvocabulary.memoryRequirementsType.gpu=GPU +controlledvocabulary.memoryRequirementsType.npu=NPU +controlledvocabulary.memoryRequirementsUnit.kb=KB +controlledvocabulary.memoryRequirementsUnit.mb=MB +controlledvocabulary.memoryRequirementsUnit.gb=GB +controlledvocabulary.memoryRequirementsUnit.tb=TB +controlledvocabulary.storageRequirementsUnit.kb=KB +controlledvocabulary.storageRequirementsUnit.mb=MB +controlledvocabulary.storageRequirementsUnit.gb=GB +controlledvocabulary.storageRequirementsUnit.tb=TB +controlledvocabulary.storageRequirementsUnit.pb=PB From 415be061f2540a9b8ac24b5af3d46a3578d889c6 Mon Sep 17 00:00:00 2001 From: Dorothea Iglezakis Date: Tue, 11 Feb 2025 17:33:34 +0100 Subject: [PATCH 6/6] included SQL statement to identify metadata entries to be migrated, added improvments from PR#25 --- .../11087-codemeta-block-improvement.md | 17 ++++++++++++----- 1 file changed, 12 insertions(+), 5 deletions(-) diff --git a/doc/release-notes/11087-codemeta-block-improvement.md b/doc/release-notes/11087-codemeta-block-improvement.md index 504447ff673..21b5ec1a598 100644 --- a/doc/release-notes/11087-codemeta-block-improvement.md +++ b/doc/release-notes/11087-codemeta-block-improvement.md @@ -1,6 +1,13 @@ -The experimental codemeta metadatablock is improved by: -* Adding subfields for size, unit and type to memoryRequirements and subfields for size and unit for storage to storageRequirements of software to improve the machine actionability of these metadata fields and enable external tools like Jupyter labs to run the software in an appropriate environmen -* adding a new subfield InfoUrl to softwareSuggestions and softwareRequirements to distinguish between the download url of a dependency (URL) and an information page of a dependency (InfoUrl) -* adjusting the termURI of the contIntegration metadata field to the changes with CodeMeta v.3.0 +### CodeMeta v3.0 -Please note, that existing metadata contents of the fields memoryRequirements and storageRequirements have to be manually migrated to the new subfields. \ No newline at end of file +The experimental CodeMeta metadata block has been improved by: + +* Adding subfields for size, unit and type to memoryRequirements and subfields for size and unit to storageRequirements of software to improve the machine actionability of these metadata fields and enable external tools like Jupyter Lab to run the software in an appropriate environment. +* Adding a new subfield InfoUrl to softwareSuggestions and softwareRequirements to distinguish between the download URL of a dependency (URL) and an information page of a dependency (InfoUrl). +* Adjusting the termURI of the contIntegration metadata field to the changes with CodeMeta v3.0. + +Please note that existing metadata contents of the fields memoryRequirements and storageRequirements have to be manually migrated to the new subfields. The following SQL query can help you identify these fields: + +`select dvo.identifier, dt.name as name, dfv.value as val from datasetfield as df, datasetfieldtype as dt, datasetfieldvalue as dfv, dvobject as dvo, datasetversion as dv where df.id = dfv.datasetfield_id and df.datasetfieldtype_id = dt.id and dvo.id = dv.dataset_id and df.datasetversion_id = dv.id and name IN ('memoryRequirements', 'storageRequirements');` + +You can download the updated CodeMeta block from the [Experimental Metadata](https://dataverse-guide--11087.org.readthedocs.build/en/11087/user/appendix.html#experimental-metadata) section of the User Guide. See also #10859 and #11087. \ No newline at end of file