Dataset Explore

[djbrooke]

Dataverse has the infrastructure to support community built, file-level external tools (TwoRavens, Data Explorer). Similar infrastructure should be added at the dataset level to support the Code Ocean integration funded by the recent Sloan grant. Additionally, supporting Binderverse (#4714), SBGrid's reprocessing tool (ping @pameyer), and other future tools should be considered.

[pdurbin]

This issue represents the initial step toward integration of Dataverse with Code Ocean (Sloan grant) and hopefully related tools such as Binder and Whole Tale (community efforts) so I'm going to leave a bit of a brain dump of recent happenings.

On Tuesday during our regular community call, the Code Ocean team shared their screen and we talked through the future integration at a pretty high level. Notes can be found at https://groups.google.com/d/msg/dataverse-community/HPLziKZbOAc/q_XEqyKEBwAJ

Yesterday @djbrooke and I called in to the first meeting of the Open Science Infrastructure Working Group organized by @craig-willis from @whole-tale to discuss a variety of computation and reproducibility topics with @aprilcs (Code Ocean) @donsizemore and @tlchristian (Odum) @choldgraf and @aculich (Binder #4714) @craig-willis , @Xarthisius and @amoeba (Whole Tale #5097) and others. Notes at https://docs.google.com/document/d/1bOVWBfhOiKGU2dYoHN_Pkpv5zPI6y819UISfkTqYpoQ/edit?usp=sharing

This morning I attended a fantastic Code Ocean workshop given by @aprilcs and I'd be happy to walk anyone through her tutorial which is captured very nicely in her slides ( http://bit.ly/harvard-oa-week ), example capsule ( https://bit.ly/2NanJLc ), and git repo ( https://github.com/aprilcs/candy_trade ). I'd also like to note that she features https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/EZSJ1S as an example of a dataset that has an excellent README describing how to reproduce results using code and data in the dataset.

@mheppler and discussed this issue this morning and to me the next logical step is to make a decision on where on the dataset page the button should go that brings the user from the dataset to Code Ocean, Whole Tale, Binder, etc., sort of like how the "Explore" button can bring you to multiple external tools at the file level such as Data Explorer and Two Ravens:

[pdurbin]

@craig-willis gave us some great feedback on how he assumed the external tool manifest would allow him express to Dataverse how to compose the final URL. This is what he tried:

"queryParameters": [
  {
    "url": "{siteUrl}/api/access/datafile/{fileId}?key={apiToken}"
  }
]

Instead, for now, he'll have to compose the final URL on his side, much like how Data Explorer does. For example base_url=detailsURL.siteUrl+"/api/access/datafile/" at https://github.com/scholarsportal/Dataverse-Data-Explorer/blob/v1.0/assets/js/controllers/details.js#L481

For completeness, here's how the query parameters look in the Data Explore external tool manifest:

"queryParameters": [
   {	
    "fileId": "{fileId}"	
   },	
   {	
    "siteUrl": "{siteUrl}"	
   },	
   {	
    "key": "{apiToken}"	
   }
]

When we work on this issue #5028 we should at least consider this feedback.

[pdurbin]

This morning I attended a fantastic Code Ocean workshop

Just a heads up that the Code Ocean interface has completely changed (though the old interface I learned is still available) and is now based on @jupyterlab: https://medium.com/codeocean/new-jupyterlab-based-capsule-page-d618f34bc636

[pdurbin]

There's lots more discussion on Code Ocean on #4714 especially starting at #4714 (comment)

[pdurbin]

There's lots of excitement on Twitter about Nature Scientific Data's "Call for submissions: Reproducible data processing" announcement: https://twitter.com/mercecrosas/status/1072899669074821122