Globus: Identify and add configurable button(s) for Upload and Download actions on Dataset page

[amberleahey]

There is a need to have additional external tools added at the Dataset level. The two main areas identified where external tools are required are the 'Upload Files' and 'Download' buttons/actions.

With this particular integration (Globus), there is a need to have a configurable button for dataset editors to connect to an external tool that would allow for uploading files and submitting large file transfers via the hosted Globus endpoint and external tool. In addition, this button should be configurable to ensure data passed between the systems is reflected throughout Dataverse and conforms to all publishing workflows. The same goes for the Download process.

At the moment there aren't open frameworks in DV for supporting edit-based Dataset external tools. Let's consider making these buttons configurable for other external tools, if so how do you suggest we get started on this?

(Some other considerations for this issue:
-need to consolidate the current Upload and Download actions in the UI or at least ensure these are consistent throughout
-Still doesn't address the Dataset Creation > Upload workflows , but this can be managed with buttons and messaging
-Slight consideration of UI / UX for Globus uploaders, but need to do more (for now the existing Upload and Download buttons were chosen, but this can change))

[djbrooke]

Thanks @amberleahey. We're discussing some prioritization this week and we should be able to follow up soon after that.

One thing that I didn't bring up during our meeting is the existing workflow that we have for rsync/SSH. In this workflow, there's a similar issue of needing the Dataset created before files can be uploaded, since the rsync script needs the dataset ID. In this case, we provided a message during Dataset create that you can upload files after saving instead of doing it all on the dataset create page (see screenshot).

This was a few years ago, and this was also pre-external tools, so we'll want to review, but I thought the way we worked around the limitations in that case was interesting.

[amberleahey]

Hi everyone, I thought I'd share some ideas for which existing buttons on the Dataverse dataset pages UI could be used for this Globus integration work. We are thinking about this workflow:

Upload

1. User creates a Dataset > Under the "Files" section options > "Upload with Globus" option is presented under other upload options > during dataset creation message appears for users "To upload files using Globus, please sign up for a Globus account and download the Globus Connect Personal application to enable transfer of your local files to Dataverse. When you are ready to upload, click "Save" during the Dataset creation process and proceed to "+ Upload Files" button that will appear on the Dataset landing page. For more information about how to use Globus in Dataverse see the Dataverse Globus Tool Guide. " (suggested wording)
   

2. On the Dataset landing page, users navigate to the "+ Upload Files" button which presents the Files page window
   Link to Globus Upload Tool presented

3. User initiates the Globus Upload Tool to transfer files to Dataverse Dataset

Download

1. The Globus Tool can support folder and file level downloading and permissions, it would be nice to make the "Access Dataset" button support Globus Download (I think this button is already configurable but will need to confirm it can pass permissions and versions in all cases)

2. At the file level, the existing download icon is a dropdown, add Globus download option

I'm not sure in all cases we need to show the Globus download options, but perhaps for now if we've enabled Globus External Tool in our instance we could show it as an option. We could also configure this to only show up when a file is uploaded via Globus or when a file is over a certain size limit (perhaps based on the instance's upload size limit). In all cases, the download buttons must support all versions of a dataset and pass permissions at the file level to the Globus Tool.

Let us know if this makes sense and would work!

[lubitchv]

File level external tool has a mandatory field contentType in manifest json. But globus file level external tool for download should work on any type of file. So there is a need either to make contentType not to be mandatory (like in dataset level external tool) and show Globus tool for any type of file or Globus Tool would have to use a different framework (not the present external tool one).

[pacian]

Hello, just trying to get a sense of where this issue is in the pipeline. I see this as a potential deliverable in Q1 2022; Q1 2022 has passed; will this be delivered in FY 2022?

[qqmyers]

Working on this through June with the intent to have it working (if a Dataverse instance configures it) for public files. Hoping to coordinate through the community slack on this.

[pdurbin]

This seems to be the issue where we are tracking the Globus UI work. I just merged the Globus PR (#8891) and deployed the code from develop to the test server @qqmyers set up (thanks!).

Here are screenshots from the following scenario:

• Transfer an XML file from a public Globus endpoint into a Dataverse dataset.
• Transfer the XML file from a Dataverse dataset to a laptop running Globus Personal Connect.