TRSA (Trusted Remote Storage Agent) and variable-level metadata upload

[akio-sone]

Related issues

• Provide API and internal framework for modifying variable-level (tabular) metadata. Provide API and internal framework for modifying variable-level (tabular) metadata. #4448
• New API endpoints for TRSA (Trusted Remote Storage Agent) of ImPACT project: stage 1 New API endpoints for TRSA (Trusted Remote Storage Agent) of ImPACT project: stage 1 #4750

Related Documents about TRSA

permission required

• https://github.com/RENCI-NRIG/impact-private/tree/master/implementation/modules/trsa
• https://docs.google.com/document/d/1NqJ9RxciS0Wfu1JSZWDesZD02m9pAeTeMuwoxDsSj04/edit
• https://docs.google.com/document/d/194Tgwy5TE0onEbr8cEwTmwfJ0FnkzNhGlqoBKOBqVGg/edit

public

• ImPACT(Infrastructure for Privacy-Assured CompuTations) Project
  http://cyberimpact.us/
• ImPACT Project: Blog
  http://cyberimpact.us/blog/

[pdurbin]

@akio-sone @jonc1438 I mentioned you over at #4821 (comment) but I wanted to highlight that for Make Data Count, we plan to only count downloads that go through Glassfish which means that downloads from TRSA, downloads from rsync, and downloads directly from Swift won't be counted.

[jonc1438]

Phil Thanks that is defiantly something we have thought about in TRSA. We for sure will need away to track any of the remote downloads or execution. Right now TRSA is planned to just to be a registration and local management system for the remote trusted data store. The user access point will always be from Dataverse and Glassfish will know about the parts in Dataverse. In our Impact Model once you leave Dataverse the Notary Service and the SAFE systems tracks data usage for downloads and access. It would be good to track the count of references/forwards to outside applications in Dataverse. Where they be Two Ravens/DDI Explore/ImPact Notary service/CodeOcean etc Jon Jonathan Crabtree Director Cyberinfrastructure Odum Institute UNC Chapel Hill www.odum.unc.edu Jonathan_Crabtree@unc.edu<mailto:Jonathan_Crabtree@unc.edu> 919-962-0517 Office 919-428-6112 Cell From: Philip Durbin <notifications@github.com> Reply-To: IQSS/dataverse <reply@reply.github.com> Date: Thursday, November 29, 2018 at 5:13 AM To: IQSS/dataverse <dataverse@noreply.github.com> Cc: Jonathan Crabtree <jonathan_crabtree@unc.edu>, Mention <mention@noreply.github.com> Subject: Re: [IQSS/dataverse] TRSA (Trusted Remote Storage Agent) and variable-level metadata upload (#5213) @akio-sone<https://github.com/akio-sone> @jonc1438<https://github.com/jonc1438> I mentioned you over at #4821 (comment)<#4821 (comment)> but I wanted to highlight that for Make Data Count, we plan to only count downloads that go through Glassfish which means that downloads from TRSA, downloads from rsync, and downloads directly from Swift won't be counted. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#5213 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AHYP4o76h4eJrDqcp_3KQms2Q3VFMhfEks5uzvx6gaJpZM4Xuh8y>.

[pdurbin]

@jonc1438 ah, ok, so it sounds like data from a TRSA will still go through Glassfish. This is sort of like how an S3 download works. The user interacts with Glassfish so we'll count the download even if the file ultimately gets downloaded directly from S3 (if dataverse.files.s3-download-redirect is set to true). Thanks!

[pdurbin]

@jonc1438 @akio-sone @donsizemore great meeting this week. Would it be helpful if I stubbed out a workflow diagram similar to the one at http://guides.dataverse.org/en/4.13/admin/make-data-count.html#architecture ? I know you have a nice diagrams at http://cyberimpact.us/architecture-overview/ and http://cyberimpact.us/dataverse-trusted-remote-storage-agent-update/ but @kcondon and I were talking about a diagram that's a little lower level, about communication back and forth between the different components, like that Make Data Count diagram. Please let me know. Thanks.

[djbrooke]

@jonc1438 @akio-sone @donsizemore @pdurbin please coordinate this with @scolapasta. Thanks!

[jonc1438]

Be happy to Jon Get Outlook for iOS<https://aka.ms/o0ukef>

…

________________________________ From: Philip Durbin <notifications@github.com> Sent: Friday, May 3, 2019 2:03 PM To: IQSS/dataverse Cc: Crabtree, Jonathan David; Mention Subject: Re: [IQSS/dataverse] TRSA (Trusted Remote Storage Agent) and variable-level metadata upload (#5213) @jonc1438<https://github.com/jonc1438> @akio-sone<https://github.com/akio-sone> @donsizemore<https://github.com/donsizemore> great meeting this week. Would it be helpful if I stubbed out a workflow diagram similar to the one at http://guides.dataverse.org/en/4.13/admin/make-data-count.html#architecture ? I know you have a nice diagrams at http://cyberimpact.us/architecture-overview/ and http://cyberimpact.us/dataverse-trusted-remote-storage-agent-update/ but @kcondon<https://github.com/kcondon> and I were talking about a diagram that's a little lower level, about communication back and forth between the different components, like that Make Data Count diagram. Please let me know. Thanks. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#5213 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AB3A7YRJS2DMDN23MB7QLRLPTR45FANCNFSM4F52D4ZA>.

[scolapasta]

@pdurbin @jonc1438 I think a diagram could help us wrap our thoughts around this for our future discussions. Once you get started, let me know, and I can help add my understanding.