From 7330b26ec2c323bac47e4aabdfd31fbe5cf7ec2f Mon Sep 17 00:00:00 2001 From: Danny Brooke Date: Wed, 29 Sep 2021 15:24:52 -0400 Subject: [PATCH 01/16] out with the old content... --- doc/release-notes/5.7-release-notes.md | 92 ++++++++++++++++++++++++++ 1 file changed, 92 insertions(+) create mode 100644 doc/release-notes/5.7-release-notes.md diff --git a/doc/release-notes/5.7-release-notes.md b/doc/release-notes/5.7-release-notes.md new file mode 100644 index 00000000000..57270b8ed28 --- /dev/null +++ b/doc/release-notes/5.7-release-notes.md @@ -0,0 +1,92 @@ +# Dataverse Software 5.7 + +This release brings new features, enhancements, and bug fixes to the Dataverse Software. Thank you to all of the community members who contributed code, suggestions, bug reports, and other assistance across the project. + +## Release Highlights + +### xxxx + +xxx + +## Major Use Cases + +Newly-supported major use cases in this release include: + +- xxx. (Issue #1724, PR #7908) + +## Notes for Dataverse Installation Administrators + +### xxx + +xxx + +## New JVM Options and DB Settings + +:xxx can be xxx + +## Notes for Tool Developers and Integrators + +### xxx + +xxx + +## Complete List of Changes + +For the complete list of code changes in this release, see the [5.7 Milestone](https://github.com/IQSS/dataverse/milestone/98?closed=1) in Github. + +For help with upgrading, installing, or general questions please post to the [Dataverse Community Google Group](https://groups.google.com/forum/#!forum/dataverse-community) or email support@dataverse.org. + +## Installation + +If this is a new installation, please see our [Installation Guide](https://guides.dataverse.org/en/5.6/installation/). + +## Upgrade Instructions + +0\. These instructions assume that you've already successfully upgraded from Dataverse Software 4.x to Dataverse Software 5 following the instructions in the [Dataverse Software 5 Release Notes](https://github.com/IQSS/dataverse/releases/tag/v5.0). After upgrading from the 4.x series to 5.0, you should progress through the other 5.x releases before attempting the upgrade to 5.7. + +The steps below include a required upgrade to Payara 5.2021.5 or higher. (It is a simple matter of reusing your existing domain directory with the new distribution). But we also recommend that you review the Payara upgrade instructions as it could be helpful during any troubleshooting: [Payara documentation](https://docs.payara.fish/community/docs/5.2021.5/documentation/user-guides/upgrade-payara.html) + +If you are running Payara as a non-root user (and you should be!), **remember not to execute the commands below as root**. Use `sudo` to change to that user first. For example, `sudo -i -u dataverse` if `dataverse` is your dedicated application user. + +In the following commands we assume that Payara 5 is installed in `/usr/local/payara5`. If not, adjust as needed. + +`export PAYARA=/usr/local/payara5` + +(or `setenv PAYARA /usr/local/payara5` if you are using a `csh`-like shell) + +1\. Undeploy the previous version + +- `$PAYARA/bin/asadmin list-applications` +- `$PAYARA/bin/asadmin undeploy dataverse<-version>` + +2\. Stop Payara + +- `service payara stop` +- `rm -rf $PAYARA/glassfish/domains/domain1/generated` + +3\. Move the current Payara directory out of the way + +- `mv $PAYARA $PAYARA.MOVED` + +4\. Download the new Payara version (5.2021.5+), and unzip it in its place + +5\. Replace the brand new payara/glassfish/domains/domain1 with your old, preserved `domain1` + +6\. In domain.xml, add/confirm the existence of the following JVM option to/in the `` section: + +`[1.8.0u251|]-Xbootclasspath/a:${com.sun.aas.installRoot}/lib/grizzly-npn-api.jar` + +7\. Start Payara + +- `service payara start` + +8\. Deploy this version. + +- `$PAYARA/bin/asadmin deploy dataverse-5.6.war` + +9\. Restart payara + +- `service payara stop` +- `service payara start` + +## Additional Release Steps From 58e2301ca27d8cb07bf3c842bc0bbb099013472d Mon Sep 17 00:00:00 2001 From: Danny Brooke Date: Wed, 29 Sep 2021 15:43:14 -0400 Subject: [PATCH 02/16] new content added in from individual files --- doc/release-notes/5.7-release-notes.md | 44 ++++++++++++++++--- doc/release-notes/6953-dev-jsf-settings.md | 4 -- doc/release-notes/7257-breadcrumbs.md | 1 - doc/release-notes/7786-ddi_changes.md | 10 ----- .../7946-external-vocab-support.md | 7 --- .../8043-skip-validate-files-on-publish.md | 7 --- doc/release-notes/8066-bag_updates.md | 3 -- .../8069-dataset_created_notices.md | 3 -- .../8073-gb-response_api_update.md | 3 -- 9 files changed, 37 insertions(+), 45 deletions(-) delete mode 100644 doc/release-notes/6953-dev-jsf-settings.md delete mode 100644 doc/release-notes/7257-breadcrumbs.md delete mode 100644 doc/release-notes/7786-ddi_changes.md delete mode 100644 doc/release-notes/7946-external-vocab-support.md delete mode 100644 doc/release-notes/8043-skip-validate-files-on-publish.md delete mode 100644 doc/release-notes/8066-bag_updates.md delete mode 100644 doc/release-notes/8069-dataset_created_notices.md delete mode 100644 doc/release-notes/8073-gb-response_api_update.md diff --git a/doc/release-notes/5.7-release-notes.md b/doc/release-notes/5.7-release-notes.md index 57270b8ed28..7d4b3ddd74a 100644 --- a/doc/release-notes/5.7-release-notes.md +++ b/doc/release-notes/5.7-release-notes.md @@ -4,21 +4,51 @@ This release brings new features, enhancements, and bug fixes to the Dataverse S ## Release Highlights -### xxxx +### Experimental Support for External Vocabulary Services -xxx +Dataverse can now be configured to associate specific metadata fields with third-party vocabulary services to provide an easy way for users to select values from those vocabularies. The mapping involves use of external Javascripts. Two such scripts have been developed so far: one for vocabularies served via the SKOSMOS protocol (see https://skosmos.org) and one allowing people to be identified via their ORCID (see https://orcid.org). The guides contain info about the new :CVocConf setting used for configuration and additional information about this functionality. Scripts, examples, and additional documentation are available at https://github.com/gdcc/dataverse-external-vocab-support. + +This functionality was initially developed by [Data Archiving and Networked Services](http://dans.knaw.nl/en/) (DANS-KNAW), the Netherlands, and funded by [SSHOC](https://sshopencloud.eu), "Social Sciences and Humanities Open Cloud". SSHOC has received funding from the European Union’s Horizon 2020 project call H2020-INFRAEOSC-04-2018, grant agreement #823782. It was further improved by the Global Dataverse Community Consortium (GDCC) and extended with the support of semantic search. + +Please watch this [online presentation](https://youtu.be/mWMPxCwB2ys?t=1794), [read the document with requirements](https://docs.google.com/document/d/1txdcFuxskRx_tLsDQ7KKLFTMR_r9IBhorDu3V_r445w/edit?ts=5fdbca9a) and join [Dataverse Working Group on Ontologies and Controlled Vocabularies](https://dataverse.org/community-calls) if you have some questions and want to contribute. ## Major Use Cases Newly-supported major use cases in this release include: -- xxx. (Issue #1724, PR #7908) +- Datasets in the root Dataverse collection now show breadcrumbs that have have a link back to the root Dataverse collection. (This was the behavior for Dataverse 4.) (Issue #xxx, PR #xxx) ## Notes for Dataverse Installation Administrators -### xxx +### Administrators can now be notified when Datasets are created + +A new :SendNotificationOnDatasetCreation setting has been added. When true, administrators and curators (those who could publish the dataset) will get an email/notification when a new dataset is created. This makes it easier to track activity in a Dataverse and, for example, allow admins to follow up when users do not publish a new dataset within some period of time. + +### Skip checksum validation if over limit when publishing datasets + +When a user requests to publish a dataset, the time taken to complete the publishing process varies based on the dataset/datafile size. + +With the additional settings of :DatasetChecksumValidationSizeLimit and :DataFileChecksumValidationSizeLimit, the checksum validation can be skipped while publishing. + +If the Dataverse admin choose to set these values, its strongly recommended to have an external auditing system runs periodically to monitor the integrity of the files in their Dataverse. + +### Update Retrieve Guestbook Response API to remove default output file + +With this release the Retrieve Guestbook Responses for a Dataverse Collection will no longer produce a file by default. You may specify an output file by adding a -o $YOURFILENAME to the curl command. + +### Dynamic Java Server Faces Configuration Options + +This release includes a new way to easily change JSF settings via MicroProfile Config, especially useful during development. +See the [development guide on "Debugging"](https://guides.dataverse.org/en/5.7/developers/debugging.html). + +### Enhancements to DDI Metadata Exports + +Several changes have been made to the DDI exports to improve support for internationalization and to improve compliance with CESSDA requirements. These changes include: + +* Addition of xml:lang attributes specifying the dataset metadata language at the document level and for individual elements such as title and description +* Specification of controlled vocabulary terms in duplicate elements in multiple languages (in the installation default langauge and, if different, the dataset metadata language) -xxx +While these changes are intended to improve harvesting and integration with external systems, the could break existing connections that make assumptions about the elements and attributes that have been changed. ## New JVM Options and DB Settings @@ -26,9 +56,9 @@ xxx ## Notes for Tool Developers and Integrators -### xxx +### Bag update to support file paths -xxx +The original Bag generation code stored all dataset files directly under the /data directory. With the addition in Dataverse of a directory path for files and then a change to allow files with different paths to have the same name, archival Bags will now use the directory path from Dataverse to avoid name collisions within the /data directory. Prior to this update, Bags from Datasets with multiple files with the same name would have been created with only one of the files with that name (with warnings in the log, but still generating the Bag). ## Complete List of Changes diff --git a/doc/release-notes/6953-dev-jsf-settings.md b/doc/release-notes/6953-dev-jsf-settings.md deleted file mode 100644 index 2ad8c45edf1..00000000000 --- a/doc/release-notes/6953-dev-jsf-settings.md +++ /dev/null @@ -1,4 +0,0 @@ -### Dynamic Java Server Faces Configuration Options - -This release includes a new way to easily change JSF settings via MicroProfile Config, especially useful during development. -See the [development guide on "Debugging"](https://guides.dataverse.org/en/latest/developers/debugging.html). diff --git a/doc/release-notes/7257-breadcrumbs.md b/doc/release-notes/7257-breadcrumbs.md deleted file mode 100644 index e49f781b6e2..00000000000 --- a/doc/release-notes/7257-breadcrumbs.md +++ /dev/null @@ -1 +0,0 @@ -Datasets in the root Dataverse collection now show breadcrumbs that have have a link back to the root Dataverse collection. (This was the behavior for Dataverse 4.) diff --git a/doc/release-notes/7786-ddi_changes.md b/doc/release-notes/7786-ddi_changes.md deleted file mode 100644 index 6cbc1cfb17c..00000000000 --- a/doc/release-notes/7786-ddi_changes.md +++ /dev/null @@ -1,10 +0,0 @@ -Note: These notes cover several related PRs (#7958, #7959 respectively for the three bullets below.) If some are not merged before the next release, these notes should be adjusted. - -### Enhancements to DDI Metadata Exports - -Several changes have been made to the DDI exports to improve support for internationalization and to improve compliance with CESSDA requirements. These changes include: - -* Addition of xml:lang attributes specifying the dataset metadata language at the document level and for individual elements such as title and description -* Specification of controlled vocabulary terms in duplicate elements in multiple languages (in the installation default langauge and, if different, the dataset metadata language) - -While these changes are intended to improve harvesting and integration with external systems, the could break existing connections that make assumptions about the elements and attributes that have been changed. diff --git a/doc/release-notes/7946-external-vocab-support.md b/doc/release-notes/7946-external-vocab-support.md deleted file mode 100644 index 25c82d1bb1d..00000000000 --- a/doc/release-notes/7946-external-vocab-support.md +++ /dev/null @@ -1,7 +0,0 @@ -### Experimental Support for External Vocabulary Services - -This functionality was initially developed by [Data Archiving and Networked Services](http://dans.knaw.nl/en/) (DANS-KNAW), the Netherlands, and funded by [SSHOC](https://sshopencloud.eu), "Social Sciences and Humanities Open Cloud". SSHOC has received funding from the European Union’s Horizon 2020 project call H2020-INFRAEOSC-04-2018, grant agreement #823782. It was further improved by the Global Dataverse Community Consortium (GDCC) and extended with the support of semantic search. - -Please watch this [online presentation](https://youtu.be/mWMPxCwB2ys?t=1794), [read the document with requirements](https://docs.google.com/document/d/1txdcFuxskRx_tLsDQ7KKLFTMR_r9IBhorDu3V_r445w/edit?ts=5fdbca9a) and join [Dataverse Working Group on Ontologies and Controlled Vocabularies](https://dataverse.org/community-calls) if you have some questions and want to contribute. - -Dataverse can now be configured to associate specific metadata fields with third-party vocabulary services to provide an easy way for users to select values from those vocabularies. The mapping involves use of external Javascripts. Two such scripts have been developed so far: one for vocabularies served via the SKOSMOS protocol (see https://skosmos.org) and one allowing people to be identified via their ORCID (see https://orcid.org). The guides contain info about the new :CVocConf setting used for configuration and additional information about this functionality. Scripts, examples, and additional documentation are available at https://github.com/gdcc/dataverse-external-vocab-support. diff --git a/doc/release-notes/8043-skip-validate-files-on-publish.md b/doc/release-notes/8043-skip-validate-files-on-publish.md deleted file mode 100644 index 01a0d35b40e..00000000000 --- a/doc/release-notes/8043-skip-validate-files-on-publish.md +++ /dev/null @@ -1,7 +0,0 @@ -### Skip checksum validation if over limit when publishing datasets - -When a user requests to publish a dataset, the time taken to complete the publishing process varies based on the dataset/datafile size. - -With the additional settings of :DatasetChecksumValidationSizeLimit and :DataFileChecksumValidationSizeLimit, the checksum validation can be skipped while publishing. - -If the Dataverse admin choose to set these values, its strongly recommended to have an external auditing system runs periodically to monitor the integrity of the files in their Dataverse. \ No newline at end of file diff --git a/doc/release-notes/8066-bag_updates.md b/doc/release-notes/8066-bag_updates.md deleted file mode 100644 index daf669fd313..00000000000 --- a/doc/release-notes/8066-bag_updates.md +++ /dev/null @@ -1,3 +0,0 @@ -### Bag update to support file paths - -The original Bag generation code stored all dataset files directly under the /data directory. With the addition in Dataverse of a directory path for files and then a change to allow files with different paths to have the same name, archival Bags will now use the directory path from Dataverse to avoid name collisions within the /data directory. Prior to this update, Bags from Datasets with multiple files with the same name would have been created with only one of the files with that name (with warnings in the log, but still generating the Bag). \ No newline at end of file diff --git a/doc/release-notes/8069-dataset_created_notices.md b/doc/release-notes/8069-dataset_created_notices.md deleted file mode 100644 index f07a4b84508..00000000000 --- a/doc/release-notes/8069-dataset_created_notices.md +++ /dev/null @@ -1,3 +0,0 @@ -### Administrators can now be notified when Datasets are created - -A new :SendNotificationOnDatasetCreation setting has been added. When true, administrators and curators (those who could publish the dataset) will get an email/notification when a new dataset is created. This makes it easier to track activity in a Dataverse and, for example, allow admins to follow up when users do not publish a new dataset within some period of time. \ No newline at end of file diff --git a/doc/release-notes/8073-gb-response_api_update.md b/doc/release-notes/8073-gb-response_api_update.md deleted file mode 100644 index ef238c5f35b..00000000000 --- a/doc/release-notes/8073-gb-response_api_update.md +++ /dev/null @@ -1,3 +0,0 @@ -### Update Retrieve Guestbook Response API to remove default output file - -With this release the Retrieve Guestbook Responses for a Dataverse Collection will no longer produce a file by default. You may specify an output file by adding a -o $YOURFILENAME to the curl command. From 7a439098904e192ac89168477cfa47d51b479d0c Mon Sep 17 00:00:00 2001 From: Danny Brooke Date: Thu, 30 Sep 2021 13:03:11 -0400 Subject: [PATCH 03/16] starting to add use cases --- doc/release-notes/5.7-release-notes.md | 21 +++++++++++++++------ 1 file changed, 15 insertions(+), 6 deletions(-) diff --git a/doc/release-notes/5.7-release-notes.md b/doc/release-notes/5.7-release-notes.md index 7d4b3ddd74a..0aa81c608ad 100644 --- a/doc/release-notes/5.7-release-notes.md +++ b/doc/release-notes/5.7-release-notes.md @@ -6,17 +6,26 @@ This release brings new features, enhancements, and bug fixes to the Dataverse S ### Experimental Support for External Vocabulary Services -Dataverse can now be configured to associate specific metadata fields with third-party vocabulary services to provide an easy way for users to select values from those vocabularies. The mapping involves use of external Javascripts. Two such scripts have been developed so far: one for vocabularies served via the SKOSMOS protocol (see https://skosmos.org) and one allowing people to be identified via their ORCID (see https://orcid.org). The guides contain info about the new :CVocConf setting used for configuration and additional information about this functionality. Scripts, examples, and additional documentation are available at https://github.com/gdcc/dataverse-external-vocab-support. - -This functionality was initially developed by [Data Archiving and Networked Services](http://dans.knaw.nl/en/) (DANS-KNAW), the Netherlands, and funded by [SSHOC](https://sshopencloud.eu), "Social Sciences and Humanities Open Cloud". SSHOC has received funding from the European Union’s Horizon 2020 project call H2020-INFRAEOSC-04-2018, grant agreement #823782. It was further improved by the Global Dataverse Community Consortium (GDCC) and extended with the support of semantic search. +Dataverse can now be configured to associate specific metadata fields with third-party vocabulary services to provide an easy way for users to select values from those vocabularies. The mapping involves use of external Javascripts. Two such scripts have been developed so far: one for vocabularies served via the [SKOSMOS protocol](https://skosmos.org) and one allowing people to be identified via their [ORCID](https://orcid.org). The guides contain info about the new :CVocConf setting used for configuration and additional information about this functionality. Scripts, examples, and additional documentation are available at the [GDCC GitHub Repository](https://github.com/gdcc/dataverse-external-vocab-support). Please watch this [online presentation](https://youtu.be/mWMPxCwB2ys?t=1794), [read the document with requirements](https://docs.google.com/document/d/1txdcFuxskRx_tLsDQ7KKLFTMR_r9IBhorDu3V_r445w/edit?ts=5fdbca9a) and join [Dataverse Working Group on Ontologies and Controlled Vocabularies](https://dataverse.org/community-calls) if you have some questions and want to contribute. +This functionality was initially developed by [Data Archiving and Networked Services](http://dans.knaw.nl/en/) (DANS-KNAW), the Netherlands, and funded by [SSHOC](https://sshopencloud.eu), "Social Sciences and Humanities Open Cloud". SSHOC has received funding from the European Union’s Horizon 2020 project call H2020-INFRAEOSC-04-2018, grant agreement #823782. It was further improved by the Global Dataverse Community Consortium (GDCC) and extended with the support of semantic search. + ## Major Use Cases Newly-supported major use cases in this release include: -- Datasets in the root Dataverse collection now show breadcrumbs that have have a link back to the root Dataverse collection. (This was the behavior for Dataverse 4.) (Issue #xxx, PR #xxx) +- Administrators will be able to set up integrations with external vocabulary services, allowing for autocomplete-assisted metadata entry, metadata standardization, and better integration with other systems. (Issue #7711, PR #7946) +- Users viewing datasets in the root Dataverse collection will now see breadcrumbs that have have a link back to the root Dataverse collection (Issue #7527, PR #8078) +- Users will be able to more easily differentiate between datasets and files through new iconography (Issue #7991, PR #8021) +- Users retrieving large guestbooks over the API will experience fewer failures (Issue #8073, PR #8084) +- (PR #7948) +- (PR #7959) +- (PR 8033) +- (PR 8067) +- (PR 8070) +- (PR 8074) ## Notes for Dataverse Installation Administrators @@ -45,8 +54,8 @@ See the [development guide on "Debugging"](https://guides.dataverse.org/en/5.7/d Several changes have been made to the DDI exports to improve support for internationalization and to improve compliance with CESSDA requirements. These changes include: -* Addition of xml:lang attributes specifying the dataset metadata language at the document level and for individual elements such as title and description -* Specification of controlled vocabulary terms in duplicate elements in multiple languages (in the installation default langauge and, if different, the dataset metadata language) +- Addition of xml:lang attributes specifying the dataset metadata language at the document level and for individual elements such as title and description +- Specification of controlled vocabulary terms in duplicate elements in multiple languages (in the installation default langauge and, if different, the dataset metadata language) While these changes are intended to improve harvesting and integration with external systems, the could break existing connections that make assumptions about the elements and attributes that have been changed. From 43f4c1ff6a35df283a3f944dbd4f967fbb5b9e7a Mon Sep 17 00:00:00 2001 From: Danny Brooke Date: Thu, 30 Sep 2021 14:28:19 -0400 Subject: [PATCH 04/16] additional use cases, start of new settinhgs --- doc/release-notes/5.7-release-notes.md | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/doc/release-notes/5.7-release-notes.md b/doc/release-notes/5.7-release-notes.md index 0aa81c608ad..380e0fd2572 100644 --- a/doc/release-notes/5.7-release-notes.md +++ b/doc/release-notes/5.7-release-notes.md @@ -8,7 +8,7 @@ This release brings new features, enhancements, and bug fixes to the Dataverse S Dataverse can now be configured to associate specific metadata fields with third-party vocabulary services to provide an easy way for users to select values from those vocabularies. The mapping involves use of external Javascripts. Two such scripts have been developed so far: one for vocabularies served via the [SKOSMOS protocol](https://skosmos.org) and one allowing people to be identified via their [ORCID](https://orcid.org). The guides contain info about the new :CVocConf setting used for configuration and additional information about this functionality. Scripts, examples, and additional documentation are available at the [GDCC GitHub Repository](https://github.com/gdcc/dataverse-external-vocab-support). -Please watch this [online presentation](https://youtu.be/mWMPxCwB2ys?t=1794), [read the document with requirements](https://docs.google.com/document/d/1txdcFuxskRx_tLsDQ7KKLFTMR_r9IBhorDu3V_r445w/edit?ts=5fdbca9a) and join [Dataverse Working Group on Ontologies and Controlled Vocabularies](https://dataverse.org/community-calls) if you have some questions and want to contribute. +Please watch the [online presentation](https://youtu.be/mWMPxCwB2ys?t=1794), [read the document with requirements](https://docs.google.com/document/d/1txdcFuxskRx_tLsDQ7KKLFTMR_r9IBhorDu3V_r445w/edit?ts=5fdbca9a) and join the [Dataverse Working Group on Ontologies and Controlled Vocabularies](https://dataverse.org/community-calls) if you have some questions and want to contribute. This functionality was initially developed by [Data Archiving and Networked Services](http://dans.knaw.nl/en/) (DANS-KNAW), the Netherlands, and funded by [SSHOC](https://sshopencloud.eu), "Social Sciences and Humanities Open Cloud". SSHOC has received funding from the European Union’s Horizon 2020 project call H2020-INFRAEOSC-04-2018, grant agreement #823782. It was further improved by the Global Dataverse Community Consortium (GDCC) and extended with the support of semantic search. @@ -20,12 +20,10 @@ Newly-supported major use cases in this release include: - Users viewing datasets in the root Dataverse collection will now see breadcrumbs that have have a link back to the root Dataverse collection (Issue #7527, PR #8078) - Users will be able to more easily differentiate between datasets and files through new iconography (Issue #7991, PR #8021) - Users retrieving large guestbooks over the API will experience fewer failures (Issue #8073, PR #8084) -- (PR #7948) -- (PR #7959) -- (PR 8033) -- (PR 8067) -- (PR 8070) -- (PR 8074) +- Administrators will now be able to set a language for metadata exports at the document or element level (Issue #7388, PR #7958) +- Administrators will now be able to specify the language of controlled vocabulary items (Issue #6751, PR #7959) +- Administrators and curators can now receive notifications when a dataset is published (Issue #8069, PR #8070) +- Administrators with large files in their installation can disable the automatic checksum verification process at publish time (Issue #8043, PR #8074) ## Notes for Dataverse Installation Administrators @@ -57,11 +55,12 @@ Several changes have been made to the DDI exports to improve support for interna - Addition of xml:lang attributes specifying the dataset metadata language at the document level and for individual elements such as title and description - Specification of controlled vocabulary terms in duplicate elements in multiple languages (in the installation default langauge and, if different, the dataset metadata language) -While these changes are intended to improve harvesting and integration with external systems, the could break existing connections that make assumptions about the elements and attributes that have been changed. +While these changes are intended to improve harvesting and integration with external systems, they could break existing connections that make assumptions about the elements and attributes that have been changed. ## New JVM Options and DB Settings -:xxx can be xxx +:SendNotificationOnDatasetCreation +:DatasetChecksumValidationSizeLimit :DataFileChecksumValidationSizeLimit ## Notes for Tool Developers and Integrators From 95a4a797739bbf5a7a8f4c2b9f3d0159e750155a Mon Sep 17 00:00:00 2001 From: Danny Brooke Date: Fri, 1 Oct 2021 16:21:45 -0400 Subject: [PATCH 05/16] update to release notes and native API doc --- doc/release-notes/5.7-release-notes.md | 12 ++++++------ doc/sphinx-guides/source/api/native-api.rst | 2 +- 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/doc/release-notes/5.7-release-notes.md b/doc/release-notes/5.7-release-notes.md index 380e0fd2572..1956d8ffe4c 100644 --- a/doc/release-notes/5.7-release-notes.md +++ b/doc/release-notes/5.7-release-notes.md @@ -22,14 +22,14 @@ Newly-supported major use cases in this release include: - Users retrieving large guestbooks over the API will experience fewer failures (Issue #8073, PR #8084) - Administrators will now be able to set a language for metadata exports at the document or element level (Issue #7388, PR #7958) - Administrators will now be able to specify the language of controlled vocabulary items (Issue #6751, PR #7959) -- Administrators and curators can now receive notifications when a dataset is published (Issue #8069, PR #8070) +- Administrators and curators can now receive notifications when a dataset is created (Issue #8069, PR #8070) - Administrators with large files in their installation can disable the automatic checksum verification process at publish time (Issue #8043, PR #8074) ## Notes for Dataverse Installation Administrators -### Administrators can now be notified when Datasets are created +### Dataset Creation Notifications for Administrators -A new :SendNotificationOnDatasetCreation setting has been added. When true, administrators and curators (those who could publish the dataset) will get an email/notification when a new dataset is created. This makes it easier to track activity in a Dataverse and, for example, allow admins to follow up when users do not publish a new dataset within some period of time. +A new :SendNotificationOnDatasetCreation setting has been added. When true, administrators and curators (those who can publish the dataset) will get an email/notification when a new dataset is created. This makes it easier to track activity in a Dataverse and, for example, allow admins to follow up when users do not publish a new dataset within some period of time. ### Skip checksum validation if over limit when publishing datasets @@ -37,13 +37,13 @@ When a user requests to publish a dataset, the time taken to complete the publis With the additional settings of :DatasetChecksumValidationSizeLimit and :DataFileChecksumValidationSizeLimit, the checksum validation can be skipped while publishing. -If the Dataverse admin choose to set these values, its strongly recommended to have an external auditing system runs periodically to monitor the integrity of the files in their Dataverse. +If the Dataverse administrator chooses to set these values, it's strongly recommended to have an external auditing system run periodically in order to monitor the integrity of the files in the Dataverse installation. ### Update Retrieve Guestbook Response API to remove default output file -With this release the Retrieve Guestbook Responses for a Dataverse Collection will no longer produce a file by default. You may specify an output file by adding a -o $YOURFILENAME to the curl command. +With this release the Retrieve Guestbook Responses for a Dataverse Collection API will no longer produce a file by default. You may specify an output file by adding a -o $YOURFILENAME to the curl command. -### Dynamic Java Server Faces Configuration Options +### Dynamic JavaServer Faces Configuration Options This release includes a new way to easily change JSF settings via MicroProfile Config, especially useful during development. See the [development guide on "Debugging"](https://guides.dataverse.org/en/5.7/developers/debugging.html). diff --git a/doc/sphinx-guides/source/api/native-api.rst b/doc/sphinx-guides/source/api/native-api.rst index 952f2c28238..ef699d8a5bd 100644 --- a/doc/sphinx-guides/source/api/native-api.rst +++ b/doc/sphinx-guides/source/api/native-api.rst @@ -587,7 +587,7 @@ You should expect a 200 ("OK") response and JSON output. Retrieve Guestbook Responses for a Dataverse Collection ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -In order to retrieve a file containing a list of Guestbook Responses in csv format for Dataverse collection, you must know either its "alias" (which the GUI calls an "identifier") or its database ID. If the Dataverse collection has more than one guestbook you may provide the id of a single guestbook as an optional parameter. If no guestbook id is provided the results returned will be the same as pressing the "Download All Responses" button on the Manage Dataset Guestbook page. If the guestbook id is provided then only those responses from that guestbook will be included. The FILENAME parameter is optional without it the responses will be displayed in the console. +In order to retrieve the Guestbook Responses for a Dataverse collection, you must know either its "alias" (which the GUI calls an "identifier") or its database ID. If the Dataverse collection has more than one guestbook you may provide the id of a single guestbook as an optional parameter. If no guestbook id is provided the results returned will be the same as pressing the "Download All Responses" button on the Manage Dataset Guestbook page. If the guestbook id is provided then only those responses from that guestbook will be included. The FILENAME parameter is optional, and if it is not included, the responses will be displayed in the console. .. note:: See :ref:`curl-examples-and-environment-variables` if you are unfamiliar with the use of ``export`` below. From 184555ce416db3850869956e298e2b76da8d20da Mon Sep 17 00:00:00 2001 From: Danny Brooke Date: Fri, 1 Oct 2021 18:07:00 -0400 Subject: [PATCH 06/16] ready for review. --- doc/release-notes/5.7-release-notes.md | 57 +++++++++----------------- 1 file changed, 20 insertions(+), 37 deletions(-) diff --git a/doc/release-notes/5.7-release-notes.md b/doc/release-notes/5.7-release-notes.md index 1956d8ffe4c..d94d0c6afee 100644 --- a/doc/release-notes/5.7-release-notes.md +++ b/doc/release-notes/5.7-release-notes.md @@ -31,7 +31,7 @@ Newly-supported major use cases in this release include: A new :SendNotificationOnDatasetCreation setting has been added. When true, administrators and curators (those who can publish the dataset) will get an email/notification when a new dataset is created. This makes it easier to track activity in a Dataverse and, for example, allow admins to follow up when users do not publish a new dataset within some period of time. -### Skip checksum validation if over limit when publishing datasets +### Skip Checksum Validation at Publish When a user requests to publish a dataset, the time taken to complete the publishing process varies based on the dataset/datafile size. @@ -39,14 +39,14 @@ With the additional settings of :DatasetChecksumValidationSizeLimit and :DataFil If the Dataverse administrator chooses to set these values, it's strongly recommended to have an external auditing system run periodically in order to monitor the integrity of the files in the Dataverse installation. -### Update Retrieve Guestbook Response API to remove default output file +### Guestbook Response API Update With this release the Retrieve Guestbook Responses for a Dataverse Collection API will no longer produce a file by default. You may specify an output file by adding a -o $YOURFILENAME to the curl command. ### Dynamic JavaServer Faces Configuration Options This release includes a new way to easily change JSF settings via MicroProfile Config, especially useful during development. -See the [development guide on "Debugging"](https://guides.dataverse.org/en/5.7/developers/debugging.html). +See the [development guide on "Debugging"](https://guides.dataverse.org/en/5.7/developers/debugging.html) for more information. ### Enhancements to DDI Metadata Exports @@ -59,12 +59,19 @@ While these changes are intended to improve harvesting and integration with exte ## New JVM Options and DB Settings -:SendNotificationOnDatasetCreation -:DatasetChecksumValidationSizeLimit :DataFileChecksumValidationSizeLimit +:SendNotificationOnDatasetCreation - A boolean setting that, if true will send an email and notification to users when a Dataset is created. Messages go to those, other than the dataset creator, who have the ability/permission necessary to publish the dataset. + +:DatasetChecksumValidationSizeLimit - disables the checksum validation while publishing for any dataset size greater than the limit. + +:DataFileChecksumValidationSizeLimit - Disables the checksum validation while publishing for any datafiles greater than the limit. + +:CVocConf - A JSON-structured setting that configures Dataverse to associate specific metadatablock fields with external vocabulary services and specific vocabularies/sub-vocabularies managed by that service. + +:MetadataLanguages - Sets which languages can be used when entering dataset metadata. ## Notes for Tool Developers and Integrators -### Bag update to support file paths +### Bags Now Support File Paths The original Bag generation code stored all dataset files directly under the /data directory. With the addition in Dataverse of a directory path for files and then a change to allow files with different paths to have the same name, archival Bags will now use the directory path from Dataverse to avoid name collisions within the /data directory. Prior to this update, Bags from Datasets with multiple files with the same name would have been created with only one of the files with that name (with warnings in the log, but still generating the Bag). @@ -76,55 +83,31 @@ For help with upgrading, installing, or general questions please post to the [Da ## Installation -If this is a new installation, please see our [Installation Guide](https://guides.dataverse.org/en/5.6/installation/). +If this is a new installation, please see our [Installation Guide](https://guides.dataverse.org/en/5.7/installation/). ## Upgrade Instructions 0\. These instructions assume that you've already successfully upgraded from Dataverse Software 4.x to Dataverse Software 5 following the instructions in the [Dataverse Software 5 Release Notes](https://github.com/IQSS/dataverse/releases/tag/v5.0). After upgrading from the 4.x series to 5.0, you should progress through the other 5.x releases before attempting the upgrade to 5.7. -The steps below include a required upgrade to Payara 5.2021.5 or higher. (It is a simple matter of reusing your existing domain directory with the new distribution). But we also recommend that you review the Payara upgrade instructions as it could be helpful during any troubleshooting: [Payara documentation](https://docs.payara.fish/community/docs/5.2021.5/documentation/user-guides/upgrade-payara.html) - -If you are running Payara as a non-root user (and you should be!), **remember not to execute the commands below as root**. Use `sudo` to change to that user first. For example, `sudo -i -u dataverse` if `dataverse` is your dedicated application user. - -In the following commands we assume that Payara 5 is installed in `/usr/local/payara5`. If not, adjust as needed. - -`export PAYARA=/usr/local/payara5` - -(or `setenv PAYARA /usr/local/payara5` if you are using a `csh`-like shell) - -1\. Undeploy the previous version +1\. Undeploy the previous version. - `$PAYARA/bin/asadmin list-applications` - `$PAYARA/bin/asadmin undeploy dataverse<-version>` -2\. Stop Payara +2\. Stop Payara and remove the generated directory - `service payara stop` - `rm -rf $PAYARA/glassfish/domains/domain1/generated` -3\. Move the current Payara directory out of the way - -- `mv $PAYARA $PAYARA.MOVED` - -4\. Download the new Payara version (5.2021.5+), and unzip it in its place - -5\. Replace the brand new payara/glassfish/domains/domain1 with your old, preserved `domain1` - -6\. In domain.xml, add/confirm the existence of the following JVM option to/in the `` section: - -`[1.8.0u251|]-Xbootclasspath/a:${com.sun.aas.installRoot}/lib/grizzly-npn-api.jar` - -7\. Start Payara +3\. Start Payara - `service payara start` -8\. Deploy this version. +4\. Deploy this version. -- `$PAYARA/bin/asadmin deploy dataverse-5.6.war` +- `$PAYARA/bin/asadmin deploy dataverse-5.7.war` -9\. Restart payara +5\. Restart payara - `service payara stop` - `service payara start` - -## Additional Release Steps From 2de79ff87aba6c92ee02c126e80a5aacedd9c392 Mon Sep 17 00:00:00 2001 From: Danny Brooke Date: Fri, 1 Oct 2021 18:09:07 -0400 Subject: [PATCH 07/16] listing --- doc/release-notes/5.7-release-notes.md | 14 +++++--------- 1 file changed, 5 insertions(+), 9 deletions(-) diff --git a/doc/release-notes/5.7-release-notes.md b/doc/release-notes/5.7-release-notes.md index d94d0c6afee..d8b7a1663ef 100644 --- a/doc/release-notes/5.7-release-notes.md +++ b/doc/release-notes/5.7-release-notes.md @@ -59,15 +59,11 @@ While these changes are intended to improve harvesting and integration with exte ## New JVM Options and DB Settings -:SendNotificationOnDatasetCreation - A boolean setting that, if true will send an email and notification to users when a Dataset is created. Messages go to those, other than the dataset creator, who have the ability/permission necessary to publish the dataset. - -:DatasetChecksumValidationSizeLimit - disables the checksum validation while publishing for any dataset size greater than the limit. - -:DataFileChecksumValidationSizeLimit - Disables the checksum validation while publishing for any datafiles greater than the limit. - -:CVocConf - A JSON-structured setting that configures Dataverse to associate specific metadatablock fields with external vocabulary services and specific vocabularies/sub-vocabularies managed by that service. - -:MetadataLanguages - Sets which languages can be used when entering dataset metadata. +- :SendNotificationOnDatasetCreation - A boolean setting that, if true will send an email and notification to users when a Dataset is created. Messages go to those, other than the dataset creator, who have the ability/permission necessary to publish the dataset. +- :DatasetChecksumValidationSizeLimit - disables the checksum validation while publishing for any dataset size greater than the limit. +- :DataFileChecksumValidationSizeLimit - Disables the checksum validation while publishing for any datafiles greater than the limit. +- :CVocConf - A JSON-structured setting that configures Dataverse to associate specific metadatablock fields with external vocabulary services and specific vocabularies/sub-vocabularies managed by that service. +- :MetadataLanguages - Sets which languages can be used when entering dataset metadata. ## Notes for Tool Developers and Integrators From b629570f93460b5bba77fb201ef0713071d2d6d1 Mon Sep 17 00:00:00 2001 From: Danny Brooke Date: Mon, 4 Oct 2021 18:21:44 -0400 Subject: [PATCH 08/16] code review feedback --- doc/release-notes/5.7-release-notes.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/doc/release-notes/5.7-release-notes.md b/doc/release-notes/5.7-release-notes.md index d8b7a1663ef..dd10464fda3 100644 --- a/doc/release-notes/5.7-release-notes.md +++ b/doc/release-notes/5.7-release-notes.md @@ -85,6 +85,14 @@ If this is a new installation, please see our [Installation Guide](https://guide 0\. These instructions assume that you've already successfully upgraded from Dataverse Software 4.x to Dataverse Software 5 following the instructions in the [Dataverse Software 5 Release Notes](https://github.com/IQSS/dataverse/releases/tag/v5.0). After upgrading from the 4.x series to 5.0, you should progress through the other 5.x releases before attempting the upgrade to 5.7. +If you are running Payara as a non-root user (and you should be!), **remember not to execute the commands below as root**. Use `sudo` to change to that user first. For example, `sudo -i -u dataverse` if `dataverse` is your dedicated application user. + +In the following commands we assume that Payara 5 is installed in `/usr/local/payara5`. If not, adjust as needed. + +`export PAYARA=/usr/local/payara5` + +(or `setenv PAYARA /usr/local/payara5` if you are using a `csh`-like shell) + 1\. Undeploy the previous version. - `$PAYARA/bin/asadmin list-applications` From c55a736450b8d1dee3e2356a560a668f3da3f2e4 Mon Sep 17 00:00:00 2001 From: Danny Brooke Date: Mon, 4 Oct 2021 18:26:21 -0400 Subject: [PATCH 09/16] code review feedback --- doc/release-notes/5.7-release-notes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/release-notes/5.7-release-notes.md b/doc/release-notes/5.7-release-notes.md index dd10464fda3..0c0a5d327cf 100644 --- a/doc/release-notes/5.7-release-notes.md +++ b/doc/release-notes/5.7-release-notes.md @@ -31,7 +31,7 @@ Newly-supported major use cases in this release include: A new :SendNotificationOnDatasetCreation setting has been added. When true, administrators and curators (those who can publish the dataset) will get an email/notification when a new dataset is created. This makes it easier to track activity in a Dataverse and, for example, allow admins to follow up when users do not publish a new dataset within some period of time. -### Skip Checksum Validation at Publish +### Skip Checksum Validation at Publish Based on Size When a user requests to publish a dataset, the time taken to complete the publishing process varies based on the dataset/datafile size. From 85e5cf45ccb546d4732035c87b0a79bb3c98f620 Mon Sep 17 00:00:00 2001 From: Danny Brooke Date: Mon, 4 Oct 2021 18:33:39 -0400 Subject: [PATCH 10/16] code review feedback --- doc/release-notes/5.7-release-notes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/release-notes/5.7-release-notes.md b/doc/release-notes/5.7-release-notes.md index 0c0a5d327cf..21939deef4e 100644 --- a/doc/release-notes/5.7-release-notes.md +++ b/doc/release-notes/5.7-release-notes.md @@ -6,7 +6,7 @@ This release brings new features, enhancements, and bug fixes to the Dataverse S ### Experimental Support for External Vocabulary Services -Dataverse can now be configured to associate specific metadata fields with third-party vocabulary services to provide an easy way for users to select values from those vocabularies. The mapping involves use of external Javascripts. Two such scripts have been developed so far: one for vocabularies served via the [SKOSMOS protocol](https://skosmos.org) and one allowing people to be identified via their [ORCID](https://orcid.org). The guides contain info about the new :CVocConf setting used for configuration and additional information about this functionality. Scripts, examples, and additional documentation are available at the [GDCC GitHub Repository](https://github.com/gdcc/dataverse-external-vocab-support). +Dataverse can now be configured to associate specific metadata fields with third-party vocabulary services to provide an easy way for users to select values from those vocabularies. The mapping involves use of external Javascripts. Two such scripts have been developed so far: one for vocabularies served via the [SKOSMOS protocol](https://skosmos.org) and one allowing people to be identified via their [ORCID](https://orcid.org). The guides contain info about the new [:CVocConf setting](https://guides.dataverse.org/en/5.7/installation/config.html#cvocconf) used for configuration and additional information about this functionality. Scripts, examples, and additional documentation are available at the [GDCC GitHub Repository](https://github.com/gdcc/dataverse-external-vocab-support). Please watch the [online presentation](https://youtu.be/mWMPxCwB2ys?t=1794), [read the document with requirements](https://docs.google.com/document/d/1txdcFuxskRx_tLsDQ7KKLFTMR_r9IBhorDu3V_r445w/edit?ts=5fdbca9a) and join the [Dataverse Working Group on Ontologies and Controlled Vocabularies](https://dataverse.org/community-calls) if you have some questions and want to contribute. From 6b01cb815e8ad7876904c2552c9020434c624970 Mon Sep 17 00:00:00 2001 From: Danny Brooke Date: Mon, 4 Oct 2021 18:35:01 -0400 Subject: [PATCH 11/16] Update doc/release-notes/5.7-release-notes.md Co-authored-by: Philip Durbin --- doc/release-notes/5.7-release-notes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/release-notes/5.7-release-notes.md b/doc/release-notes/5.7-release-notes.md index 21939deef4e..6b31106596e 100644 --- a/doc/release-notes/5.7-release-notes.md +++ b/doc/release-notes/5.7-release-notes.md @@ -59,7 +59,7 @@ While these changes are intended to improve harvesting and integration with exte ## New JVM Options and DB Settings -- :SendNotificationOnDatasetCreation - A boolean setting that, if true will send an email and notification to users when a Dataset is created. Messages go to those, other than the dataset creator, who have the ability/permission necessary to publish the dataset. +- :SendNotificationOnDatasetCreation - A boolean setting that, if true will send an email and notification to additional users when a Dataset is created. Messages go to those, other than the dataset creator, who have the ability/permission necessary to publish the dataset. - :DatasetChecksumValidationSizeLimit - disables the checksum validation while publishing for any dataset size greater than the limit. - :DataFileChecksumValidationSizeLimit - Disables the checksum validation while publishing for any datafiles greater than the limit. - :CVocConf - A JSON-structured setting that configures Dataverse to associate specific metadatablock fields with external vocabulary services and specific vocabularies/sub-vocabularies managed by that service. From 32e182fac2624b1b53e595bb0bcd3e11a87ad0d9 Mon Sep 17 00:00:00 2001 From: Danny Brooke Date: Mon, 4 Oct 2021 18:36:10 -0400 Subject: [PATCH 12/16] code review feedback --- doc/release-notes/5.7-release-notes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/release-notes/5.7-release-notes.md b/doc/release-notes/5.7-release-notes.md index 6b31106596e..ec3f9222771 100644 --- a/doc/release-notes/5.7-release-notes.md +++ b/doc/release-notes/5.7-release-notes.md @@ -29,7 +29,7 @@ Newly-supported major use cases in this release include: ### Dataset Creation Notifications for Administrators -A new :SendNotificationOnDatasetCreation setting has been added. When true, administrators and curators (those who can publish the dataset) will get an email/notification when a new dataset is created. This makes it easier to track activity in a Dataverse and, for example, allow admins to follow up when users do not publish a new dataset within some period of time. +A new :SendNotificationOnDatasetCreation setting has been added. When true, administrators and curators (those who can publish the dataset) will get a notification when a new dataset is created. This makes it easier to track activity in a Dataverse and, for example, allow admins to follow up when users do not publish a new dataset within some period of time. ### Skip Checksum Validation at Publish Based on Size From 4e4fa010565bb2d5eb107f8e2e09d788b178d334 Mon Sep 17 00:00:00 2001 From: Danny Brooke Date: Mon, 4 Oct 2021 20:19:58 -0400 Subject: [PATCH 13/16] feedback from code review --- doc/release-notes/5.7-release-notes.md | 1 + 1 file changed, 1 insertion(+) diff --git a/doc/release-notes/5.7-release-notes.md b/doc/release-notes/5.7-release-notes.md index ec3f9222771..18b1558206a 100644 --- a/doc/release-notes/5.7-release-notes.md +++ b/doc/release-notes/5.7-release-notes.md @@ -20,6 +20,7 @@ Newly-supported major use cases in this release include: - Users viewing datasets in the root Dataverse collection will now see breadcrumbs that have have a link back to the root Dataverse collection (Issue #7527, PR #8078) - Users will be able to more easily differentiate between datasets and files through new iconography (Issue #7991, PR #8021) - Users retrieving large guestbooks over the API will experience fewer failures (Issue #8073, PR #8084) +- Users will now be able to set a language for metadata entry for datasets (Issue #7388, PR #7958) - Administrators will now be able to set a language for metadata exports at the document or element level (Issue #7388, PR #7958) - Administrators will now be able to specify the language of controlled vocabulary items (Issue #6751, PR #7959) - Administrators and curators can now receive notifications when a dataset is created (Issue #8069, PR #8070) From 6df2f7ab867737c09be0deef736ad96a7e8b6f1d Mon Sep 17 00:00:00 2001 From: Danny Brooke Date: Thu, 7 Oct 2021 16:16:08 -0400 Subject: [PATCH 14/16] integrating curation labels, code review updates --- doc/release-notes/5.7-release-notes.md | 42 +++++++++++++++++++++-- doc/release-notes/7967-curation-labels.md | 35 ------------------- 2 files changed, 39 insertions(+), 38 deletions(-) delete mode 100644 doc/release-notes/7967-curation-labels.md diff --git a/doc/release-notes/5.7-release-notes.md b/doc/release-notes/5.7-release-notes.md index 18b1558206a..307573bd79f 100644 --- a/doc/release-notes/5.7-release-notes.md +++ b/doc/release-notes/5.7-release-notes.md @@ -12,6 +12,14 @@ Please watch the [online presentation](https://youtu.be/mWMPxCwB2ys?t=1794), [re This functionality was initially developed by [Data Archiving and Networked Services](http://dans.knaw.nl/en/) (DANS-KNAW), the Netherlands, and funded by [SSHOC](https://sshopencloud.eu), "Social Sciences and Humanities Open Cloud". SSHOC has received funding from the European Union’s Horizon 2020 project call H2020-INFRAEOSC-04-2018, grant agreement #823782. It was further improved by the Global Dataverse Community Consortium (GDCC) and extended with the support of semantic search. +### Curation Status Labels + +A new :AllowedCurationLabels setting allows a sysadmins to define one or more sets of labels that can be applied to a draft Dataset version via the user interface or API to indicate the status of the dataset with respect to a defined curation process. + +Labels are completely customizable (alphanumeric or spaces, up to 32 characters, e.g. "Author contacted", "Privacy Review", "Awaiting paper publication"). Superusers can select a specific set of labels, or disable this functionality per collection. Anyone who can publish a draft dataset (e.g. curators) can set/change/remove labels (from the set specified for the collection containing the dataset) via the user interface or via an API. The API also would allow external tools to search for, read and set labels on Datasets, providing an integration mechanism. Labels are visible on the Dataset page and in Dataverse collection listings/search results. Internally, the labels have no effect, and at publication, any existing label will be removed. A reporting API call allows admins to get a list of datasets and their curation statuses. + +The Solr schema must be updated as part of installing the release of Dataverse containing this feature for it to work. + ## Major Use Cases Newly-supported major use cases in this release include: @@ -20,9 +28,9 @@ Newly-supported major use cases in this release include: - Users viewing datasets in the root Dataverse collection will now see breadcrumbs that have have a link back to the root Dataverse collection (Issue #7527, PR #8078) - Users will be able to more easily differentiate between datasets and files through new iconography (Issue #7991, PR #8021) - Users retrieving large guestbooks over the API will experience fewer failures (Issue #8073, PR #8084) -- Users will now be able to set a language for metadata entry for datasets (Issue #7388, PR #7958) -- Administrators will now be able to set a language for metadata exports at the document or element level (Issue #7388, PR #7958) -- Administrators will now be able to specify the language of controlled vocabulary items (Issue #6751, PR #7959) +- Dataverse collection administrators can specify which language will be used when entering metadata for new Datasets in a collection, based on a list of languages specified by the Dataverse installation administrator. (Issue #7388, PR #7958) +- Users will see the language used for metadata entry indicated at the document or element level in metadata exports (Issue #7388, PR #7958) +- Administrators will now be able to specify the language(s) of controlled vocabulary entries, in addition to the installation's default language (Issue #6751, PR #7959) - Administrators and curators can now receive notifications when a dataset is created (Issue #8069, PR #8070) - Administrators with large files in their installation can disable the automatic checksum verification process at publish time (Issue #8043, PR #8074) @@ -116,3 +124,31 @@ In the following commands we assume that Payara 5 is installed in `/usr/local/pa - `service payara stop` - `service payara start` + +## Additional Release Steps + +1\. Replace Solr schema.xml to allow Curation Labels to be used. See specific instructions below for those installations with custom metadata blocks (1a) and those without (1b). + +1a\. + +For installations with Custom Metadata Blocks: + +-stop solr instance (usually service solr stop, depending on solr installation/OS, see the [Installation Guide](https://guides.dataverse.org/en/5.7/installation/prerequisites.html#solr-init-script) + +- add the following line to your schema.xml: + + + +- restart solr instance (usually service solr start, depending on solr/OS) + +1b\. + +For installations without Custom Metadata Blocks: + +-stop solr instance (usually service solr stop, depending on solr installation/OS, see the [Installation Guide](https://guides.dataverse.org/en/5.7/installation/prerequisites.html#solr-init-script) + +-replace schema.xml + +cp /tmp/dvinstall/schema.xml /usr/local/solr/solr-8.8.1/server/solr/collection1/conf + +-start solr instance (usually service solr start, depending on solr/OS) \ No newline at end of file diff --git a/doc/release-notes/7967-curation-labels.md b/doc/release-notes/7967-curation-labels.md deleted file mode 100644 index 2742519329c..00000000000 --- a/doc/release-notes/7967-curation-labels.md +++ /dev/null @@ -1,35 +0,0 @@ -### Curation Status Labels - -A new :AllowedCurationLabels setting allows a sysadmins to define one or more sets of labels that can be applied to a draft Dataset version via the user interface or API to indicate the status of the dataset with respect to a defined curation process. - -Labels are completely customizable (alphanumeric or spaces, up to 32 characters, e.g. "Author contacted", "Privacy Review", "Awaiting paper publication"). Superusers can select a specific set of labels, or disable this functionality per collection. Anyone who can publish a draft dataset (e.g. curators) can set/change/remove labels (from the set specified for the collection containing the dataset) via the user interface or via an API. The API also would allow external tools to search for, read and set labels on Datasets, providing an integration mechanism. Labels are visible on the Dataset page and in Dataverse collection listings/search results. Internally, the labels have no effect, and at publication, any existing label will be removed. A reporting API call allows admins to get a list of datasets and their curation statuses. - -The Solr schema must be updated as part of installing the release of Dataverse containing this feature for it to work. - -## Additional Release Steps - -1\. Replace Solr schema.xml to allow Curation Labels to be used. See specific instructions below for those installations with custom metadata blocks (1a) and those without (1b). - -1a\. - -For installations with Custom Metadata Blocks: - --stop solr instance (usually service solr stop, depending on solr installation/OS, see the [Installation Guide](https://guides.dataverse.org/en/5.7/installation/prerequisites.html#solr-init-script) - -- add the following line to your schema.xml: - - - -- restart solr instance (usually service solr start, depending on solr/OS) - -1b\. - -For installations without Custom Metadata Blocks: - --stop solr instance (usually service solr stop, depending on solr installation/OS, see the [Installation Guide](https://guides.dataverse.org/en/5.7/installation/prerequisites.html#solr-init-script) - --replace schema.xml - -cp /tmp/dvinstall/schema.xml /usr/local/solr/solr-8.8.1/server/solr/collection1/conf - --start solr instance (usually service solr start, depending on solr/OS) \ No newline at end of file From befb3ffdcd7d3477fb461df1a32f44f5ada30772 Mon Sep 17 00:00:00 2001 From: Danny Brooke Date: Thu, 7 Oct 2021 16:21:16 -0400 Subject: [PATCH 15/16] adding setting for curation labels --- doc/release-notes/5.7-release-notes.md | 1 + 1 file changed, 1 insertion(+) diff --git a/doc/release-notes/5.7-release-notes.md b/doc/release-notes/5.7-release-notes.md index 307573bd79f..b5bd590575d 100644 --- a/doc/release-notes/5.7-release-notes.md +++ b/doc/release-notes/5.7-release-notes.md @@ -73,6 +73,7 @@ While these changes are intended to improve harvesting and integration with exte - :DataFileChecksumValidationSizeLimit - Disables the checksum validation while publishing for any datafiles greater than the limit. - :CVocConf - A JSON-structured setting that configures Dataverse to associate specific metadatablock fields with external vocabulary services and specific vocabularies/sub-vocabularies managed by that service. - :MetadataLanguages - Sets which languages can be used when entering dataset metadata. +- :AllowedCurationLabels - A JSON Object containing lists of allowed labels (up to 32 characters, spaces allowed) that can be set, via API or UI by users with the permission to publish a dataset. The set of labels allowed for datasets can be selected by a superuser - via the Dataverse collection page (Edit/General Info) or set via API call. ## Notes for Tool Developers and Integrators From 7b96f4d15cc2e8ecf4e98f8eb4249e1659e25de0 Mon Sep 17 00:00:00 2001 From: Danny Brooke Date: Fri, 8 Oct 2021 09:18:33 -0400 Subject: [PATCH 16/16] updates from code review --- doc/release-notes/5.7-release-notes.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/doc/release-notes/5.7-release-notes.md b/doc/release-notes/5.7-release-notes.md index b5bd590575d..12d3dc98158 100644 --- a/doc/release-notes/5.7-release-notes.md +++ b/doc/release-notes/5.7-release-notes.md @@ -24,13 +24,13 @@ The Solr schema must be updated as part of installing the release of Dataverse c Newly-supported major use cases in this release include: -- Administrators will be able to set up integrations with external vocabulary services, allowing for autocomplete-assisted metadata entry, metadata standardization, and better integration with other systems. (Issue #7711, PR #7946) +- Administrators will be able to set up integrations with external vocabulary services, allowing for autocomplete-assisted metadata entry, metadata standardization, and better integration with other systems (Issue #7711, PR #7946) - Users viewing datasets in the root Dataverse collection will now see breadcrumbs that have have a link back to the root Dataverse collection (Issue #7527, PR #8078) - Users will be able to more easily differentiate between datasets and files through new iconography (Issue #7991, PR #8021) - Users retrieving large guestbooks over the API will experience fewer failures (Issue #8073, PR #8084) -- Dataverse collection administrators can specify which language will be used when entering metadata for new Datasets in a collection, based on a list of languages specified by the Dataverse installation administrator. (Issue #7388, PR #7958) -- Users will see the language used for metadata entry indicated at the document or element level in metadata exports (Issue #7388, PR #7958) -- Administrators will now be able to specify the language(s) of controlled vocabulary entries, in addition to the installation's default language (Issue #6751, PR #7959) +- Dataverse collection administrators can specify which language will be used when entering metadata for new Datasets in a collection, based on a list of languages specified by the Dataverse installation administrator (Issue #7388, PR #7958) + - Users will see the language used for metadata entry indicated at the document or element level in metadata exports (Issue #7388, PR #7958) + - Administrators will now be able to specify the language(s) of controlled vocabulary entries, in addition to the installation's default language (Issue #6751, PR #7959) - Administrators and curators can now receive notifications when a dataset is created (Issue #8069, PR #8070) - Administrators with large files in their installation can disable the automatic checksum verification process at publish time (Issue #8043, PR #8074)