diff --git a/doc/release-notes/3937-new-installer-script.md b/doc/release-notes/3937-new-installer-script.md deleted file mode 100644 index 7ecfe33ae88..00000000000 --- a/doc/release-notes/3937-new-installer-script.md +++ /dev/null @@ -1,12 +0,0 @@ -## New Installer Script - -We are introducing a new installer script, written in Python. It is -intended to eventually replace the old installer (written in -Perl). But for now it is being offered as an (experimental) -alternative. - -See README_python.txt in scripts/installer and/or in the installer -bundle for more information. - - - diff --git a/doc/release-notes/4.19-release-notes.md b/doc/release-notes/4.19-release-notes.md index fa8ccfafa1c..56977f5b277 100644 --- a/doc/release-notes/4.19-release-notes.md +++ b/doc/release-notes/4.19-release-notes.md @@ -1,8 +1,125 @@ # Dataverse 4.19 -Update Geospatial Metadata Block +This release brings new features, enhancements, and bug fixes to Dataverse. Thank you to all of the community members who contributed code, suggestions, bug reports, and other assistance across the project. -This update adds commas separating the values entered into Geographic Coverage. +## Release Highlights + +### Open ID Connect Support + +Dataverse now provides basic support for any OpenID Connect (OIDC) compliant authentication provider. + +Prior to supporting this standard, new authentication methods needed to be added by pull request. OIDC support provides a standardized way for authentication, sharing user information, and more. You are able to use any compliant provider just by loading a configuration file, without touching the codebase. While the usual prominent providers like Google and others feature OIDC support there are plenty of other options to easily attach your installation to a custom authentication provider, using enterprise grade software. + +See the [OpenID Connect Login Options documentation](http://guides.dataverse.org/en/4.19/installation/oidc.html) in the Installation Guide for more details. + +This is to be extended with support for attribute mapping, group syncing and more in future versions of the code. + +### Python Installer + +We are introducing a new installer script, written in Python. It is intended to eventually replace the old installer (written in Perl). For now it is being offered as an (experimental) alternative. + +See [README_python.txt](https://github.com/IQSS/dataverse/blob/v4.19/scripts/installer/README_python.txt) in scripts/installer and/or in the installer bundle for more information. + +## Major Use Cases + +Newly-supported use cases in this release include: + +- Dataverse installation administrators will be able to experiment with a Python Installer (Issue #3937, PR #6484) +- Dataverse installation administrators will be able to set up an OIDC-compliant login options by editing a configuration file and with no need for a code change (Issue #6432, PR #6433) +- Following setup by a Dataverse administration, users will be able to log in using OIDC-compliant methods (Issue #6432, PR #6433) +- Users of the Search API will see additional fields in the JSON output (Issues #6300, #6396, PR #6441) +- Users loading the support form will now be presented with the math challenge as expected and will be able to successfully send an email to support (Issue #6307, PR #6462) +- Users of https://mybinder.org can now spin up Jupyter Notebooks and other computational environments from Dataverse DOIs (Issue #4714, PR #6453) + +## Notes for Dataverse Installation Administrators + +### Security vulnerability in Solr + +A serious security issue has recently been identified in multiple versions of Solr search engine, including v.7.3 that Dataverse is currently using. Follow the instructions below to verify that your installation is safe from a potential attack. You can also consult the following link for a detailed description of the issue: + +RCE in Solr via Velocity Template. + +The vulnerability allows an intruder to execute arbitrary code on the system running Solr. Fortunately, it can only be exploited if Solr API access point is open to direct access from public networks (aka, "the outside world"), which is NOT needed in a Dataverse installation. + +We have always recommended having Solr (port 8983) firewalled off from public access in our installation guides. But we recommend that you double-check your firewall settings and verify that the port is not accessible from outside networks. The simplest quick test is to try the following URL in your browser: + + `http://:8983` + +and confirm that you get "access denied" or that it times out, etc. + +In most cases, when Solr runs on the same server as the Dataverse web application, you will only want the port accessible from localhost. We also recommend that you add the following arguments to the Solr startup command: `-j jetty.host=127.0.0.1`. This will make Solr accept connections from localhost only; adding redundancy, in case of the firewall failure. + +In a case where Solr needs to run on a different host, make sure that the firewall limits access to the port only to the Dataverse web host(s), by specific ip address(es). + +We would also like to reiterate that it is simply never a good idea to run Solr as root! Running the process as a non-privileged user would substantially minimize any potential damage even in the event that the instance is compromised. + +### Citation and Geospatial Metadata Block Updates + +We updated two metadata blocks in this release. Updating these metadata blocks is mentioned in the step-by-step upgrade instructions below. + +### Run ReExportall + +We made changes to the JSON Export in this release (#6246). If you'd like these changes to reflected in your JSON exports, you should run ReExportall as part of the upgrade process. We've included this in the step-by-step instructions below. + +### BinderHub + +https://mybinder.org now supports spinning up Jupyter Notebooks and other computational environments from Dataverse DOIs. + +### Widgets update for OpenScholar + +We updated the code for widgets so that they will keep working in OpenScholar sites after the upcoming upgrade OpenScholar upgrade to Drupal 8. If users of your dataverse have embedded widgets on an Openscholar site that upgrades to Drupal 8, you will need to run this Dataverse version (or later) for the widgets to keep working. + +### Payara tech preview + +Dataverse 4 has always run on Glassfish 4.1 but changes in this release (PR #6523) should open the door to upgrading to Payara 5 eventually. Production installations of Dataverse should remain on Glassfish 4.1 but feedback from any experiments running Dataverse on Payara 5 is welcome via the [usual channels](https://dataverse.org/contact). + +## Notes for Tool Developers and Integrators + +### Search API + +The boolean parameter `query_entities` has been removed from the Search API. The former "true" behavior of "whether entities are queried via direct database calls (for developer use)" is now always true. + +Additional fields are now available via the Search API, mostly related to information about specific dataset versions. + +## Complete List of Changes + +For the complete list of code changes in this release, see the 4.19 milestone in Github. + +For help with upgrading, installing, or general questions please post to the Dataverse Google Group or email support@dataverse.org. + +## Installation + +If this is a new installation, please see our Installation Guide. + +## Upgrade + +1. Undeploy the previous version. + +- <glassfish install path>/glassfish4/bin/asadmin list-applications +- <glassfish install path>/glassfish4/bin/asadmin undeploy dataverse + +2. Stop glassfish and remove the generated directory, start. + +- service glassfish stop +- remove the generated directory: rm -rf <glassfish install path>glassfish4/glassfish/domains/domain1/generated +- service glassfish start + +3. Deploy this version. + +- <glassfish install path>/glassfish4/bin/asadmin deploy <path>dataverse-4.19.war + +4. Restart glassfish. + +5. Update Citation Metadata Block + +- `wget https://github.com/IQSS/dataverse/releases/download/v4.19/citation.tsv` +- `curl http://localhost:8080/api/admin/datasetfield/load -X POST --data-binary @citation.tsv -H "Content-type: text/tab-separated-values"` + +6. Update Geospatial Metadata Block - `wget https://github.com/IQSS/dataverse/releases/download/v4.19/geospatial.tsv` - `curl http://localhost:8080/api/admin/datasetfield/load -X POST --data-binary @geospatial.tsv -H "Content-type: text/tab-separated-values"` + +7. (Optional) Run ReExportall to update JSON Exports + + diff --git a/doc/release-notes/4714-binder.md b/doc/release-notes/4714-binder.md deleted file mode 100644 index 9e9960636d7..00000000000 --- a/doc/release-notes/4714-binder.md +++ /dev/null @@ -1 +0,0 @@ -https://mybinder.org now supports spinning up Jupyter Notebooks from Dataverse DOIs. diff --git a/doc/release-notes/6300-6396-search-api.md b/doc/release-notes/6300-6396-search-api.md deleted file mode 100644 index f015eeff01c..00000000000 --- a/doc/release-notes/6300-6396-search-api.md +++ /dev/null @@ -1,13 +0,0 @@ -## Major Use Cases - -Newly-supported use cases in this release include: - -- Search API users will see additional fields in the JSON output #6300 #6396 - -## Notes for Tool Developers and Integrators - -### Search API - -The boolean parameter `query_entities` has been removed from the Search API. - -The former "true" behavior of "whether entities are queried via direct database calls (for developer use)" is now always true. diff --git a/doc/release-notes/6426-reexport-all b/doc/release-notes/6426-reexport-all deleted file mode 100644 index 854c188a3a5..00000000000 --- a/doc/release-notes/6426-reexport-all +++ /dev/null @@ -1,14 +0,0 @@ -## Notes for Dataverse Installation Administrators - -### Run ReExportall - -We made changes to the JSON Export in this release (#6246). If you'd like these changes to reflected in your JSON exports, you should run ReExportall as part of the upgrade process. We've included this in the detailed instructions below. - - - - - - -xxx. (Optional) Run ReExportall to update JSON Exports - - \ No newline at end of file diff --git a/doc/release-notes/6432-basic-oidc-support.md b/doc/release-notes/6432-basic-oidc-support.md deleted file mode 100644 index 415d4919d91..00000000000 --- a/doc/release-notes/6432-basic-oidc-support.md +++ /dev/null @@ -1,16 +0,0 @@ -# Basic OpenID Connect support -Working on epic #5974 brought us refactored code for our OAuth 2.0 based login options. -This has now been extended to provide basic support for any OpenID Connect compliant -authentication provider. - -While with our OAuth 2.0 login options you had to implement support for every provider -by pull request, OpenID Connect provides a standardized way for authentication, user -details and more. You are able to use any provider just by loading a configuration file, -without touching the codebase. - -While the usual prominent providers like Google et al feature OIDC support, there are -plenty of options to easily attach your current user storage to a custom made provider, -using enterprise grade software. See documentation for more details. - -This is to be extended with support for attribute mapping, group syncing and more in -future versions of the code. \ No newline at end of file diff --git a/doc/release-notes/dataverse-security-10-solr-vulnerability.md b/doc/release-notes/dataverse-security-10-solr-vulnerability.md deleted file mode 100644 index e36c3f803d2..00000000000 --- a/doc/release-notes/dataverse-security-10-solr-vulnerability.md +++ /dev/null @@ -1,39 +0,0 @@ -# Security vulnerability in Solr - -A serious security issue has recently been identified in multiple -versions of Solr search engine, including v.7.3 that Dataverse is -currently using. Follow the instructions below to verify that your -installation is safe from a potential attack. You can also consult the -following link for a detailed description of the issue: -RCE in Solr via Velocity Template. - -The vulnerability allows an intruder to execute arbitrary code on -the system running Solr. Fortunately, it can only be exploited if Solr -API access point is open to direct access from public networks (aka, -"the outside world"), which is NOT needed in a Dataverse installation. - -We have always recommended having Solr (port 8983) firewalled off from -public access in our installation guides. But we recommend that you -double-check your firewall settings and verify that the port is not -accessible from outside networks. The simplest quick test is to try -the following URL in your browser: - - `http://:8983` - -and confirm that you get "access denied" or that it times out, etc. - -In most cases, when Solr runs on the same server as the Dataverse web -application, you will only want the port accessible from localhost. We -also recommend that you add the following arguments to the Solr -startup command: `-j jetty.host=127.0.0.1`. This will make Solr accept -connections from localhost only; adding redundancy, in case of the -firewall failure. - -In a case where Solr needs to run on a different host, make sure that -the firewall limits access to the port only to the Dataverse web -host(s), by specific ip address(es). - -We would also like to reiterate that it is simply never a good idea to -run Solr as root! Running the process as a non-privileged user would -substantially minimize any potential damage even in the event that the -instance is compromised.