Migrate Collections For Dataverses Using New Dataverse Functionality

[eaquigley]

---

Author Name: Eleni Castro (@posixeleni)
Original Redmine Issue: 3530, https://redmine.hmdc.harvard.edu/issues/3530
Original Date: 2014-02-03

---

Since we are moving away from Collections for 4.0, users within a Dataverse should be able/have the option to group/organize Datasets into custom metadata blocks, which would display as a Facet when browsing.

While most Dataverses that use Collections and Subnetworks can easily be converted to Dataverses (see this: http://irri.org/tools-and-databases/irri-dataverse), we have identified a couple of Dataverses that use “Subjects” or “Journal Issues/Volumes” to form their Collections, which would not convert well into Dataverses. For example:

• MRA uses Topic Classifications as Collections
• Journal Dataverses use Issues/Volumes as Collections:
  ** RESTAT: http://thedata.harvard.edu/dvn/dv/restat
  ** AJPS: http://thedata.harvard.edu/dvn/dv/ajps
• AfricaRice: http://thedata.harvard.edu/dvn/dv/africarice
• IFPRI has some collections that could be made into Dataverses (ex. Social Accounting Matrix (SAM)) but others which resemble Topic Classifications (ex. Regional)
• ICRAF http://thedata.harvard.edu/dvn/dv/icraf

While these will involve some automated migration there will need to be a manual review of the abovementioned Dataverses to ensure that these Dataverse Collections migrated successfully to using custom metadata blocks.

For more information review meeting notes: https://docs.google.com/document/d/1SizZ0K8cj4CzyBBg7pkup_SHqD8IECOdqBg2V0pIKYY/edit?usp=sharing

[eaquigley]

---

Original Redmine Comment
Author Name: Eleni Castro (@posixeleni)
Original Date: 2014-02-03T23:45:29Z

---

Eleni Castro wrote:

Since we are moving away from Collections for 4.0, users within a Dataverse should be able/have the option to group/organize Datasets into Topic Classifications, which would display as a Facet when browsing.

While most Dataverses that use Collections and Subnetworks can easily be converted to Dataverses, we have identified a couple of Dataverses that use “Subjects” or “Journal Issues/Volumes” to form their Collections, which would not convert well into Dataverses. For example:

• MRA uses Topic Classifications as Collections
• Journal Dataverses use Issues/Volumes as Collections:
  ** RESTAT: http://thedata.harvard.edu/dvn/dv/restat
  ** AJPS: http://thedata.harvard.edu/dvn/dv/ajps
• IFPRI has some collections that could be made into Dataverses (ex. Social Accounting Matrix (SAM)) but others which resemble Topic Classifications (ex. Regional)

While these automation will involve some automated migration there will need to be a manual review to ensure that these Dataverse Collections migrated successfully to using Topic Classification.

For more information review meeting notes: https://docs.google.com/document/d/1SizZ0K8cj4CzyBBg7pkup_SHqD8IECOdqBg2V0pIKYY/edit?usp=sharing

[eaquigley]

---

Original Redmine Comment
Author Name: Eleni Castro (@posixeleni)
Original Date: 2014-02-10T18:27:11Z

---

Eleni Castro wrote:

Since we are moving away from Collections for 4.0, users within a Dataverse should be able/have the option to group/organize Datasets into Topic Classifications, which would display as a Facet when browsing.

While most Dataverses that use Collections and Subnetworks can easily be converted to Dataverses, we have identified a couple of Dataverses that use “Subjects” or “Journal Issues/Volumes” to form their Collections, which would not convert well into Dataverses. For example:

• MRA uses Topic Classifications as Collections
• Journal Dataverses use Issues/Volumes as Collections:
  ** RESTAT: http://thedata.harvard.edu/dvn/dv/restat
  ** AJPS: http://thedata.harvard.edu/dvn/dv/ajps
  *AfricaRice: http://thedata.harvard.edu/dvn/dv/africarice
• IFPRI has some collections that could be made into Dataverses (ex. Social Accounting Matrix (SAM)) but others which resemble Topic Classifications (ex. Regional)
• ICRAF http://thedata.harvard.edu/dvn/dv/icraf

While these will involve some automated migration there will need to be a manual review of the abovementioned Dataverses to ensure that these Dataverse Collections migrated successfully to using Topic Classification.

For more information review meeting notes: https://docs.google.com/document/d/1SizZ0K8cj4CzyBBg7pkup_SHqD8IECOdqBg2V0pIKYY/edit?usp=sharing

[eaquigley]

---

Original Redmine Comment
Author Name: Eleni Castro (@posixeleni)
Original Date: 2014-03-04T17:06:26Z

---

Changing the title & description of this ticket to clarify that this would not be migrated to Topic Classifications but to custom metadata blocks depending on individual Dataverse collections

[eaquigley]

@posixeleni since this is related to data migration but also has metadata involved with it, should I assign this ticket to you or @kcondon ?

[posixeleni]

This is going to be a lot of work if we are to build and sustain the usage of custom metadata blocks for each Dataverse that used collections or wants to use the concept of grouping dataverses & datasets as collections in the future.

In the long term there needs to be an end-user friendly way to organize Dataverses and Datasets in different ways that can be flexible (no version control, easy to update or remove) for our different use cases (some are listed below as examples):

1. organize dataverses & datasets thematically, categorically
2. journal datasets need to be organized by date, volume and issue number
   @eaquigley any chance you can help me brainstorm solutions for this?

However, @sbarbosadataverse we can start by seeing how/if we can display the collections for Murray using a Custom Metadata Block.

[eaquigley]

@posixeleni definitely can help brainstorm solutions!