4949 download package s3 by matthew-a-dunlap · Pull Request #5359 · IQSS/dataverse

matthew-a-dunlap · 2018-11-30T23:10:51Z

Note: There are still two changes incoming for this PR.

Documentation around the :DownloadMethod changes and a .rst page on how to set up dcm s3.
A bump to the dcm version number once this pr is merged S3 package zip sbgrid/data-capture-module#35

Related Issues

connects to Download Package File from S3 #4949

Pull Request Checklist

Unit tests completed
Integration tests: None
Deployment requirements, SQL updates, Solr updates, etc.: None
Documentation completed
Merged latest from "develop" branch and resolved conflicts

This reverts commit c8f80f0.

This move allows that code to be reused in the new package popup This also contains a bit of render logic around s3 package UI

There was a bug in the previous story where the imported dcm s3 files did not have an S3 identifier.

Wired up for all flows, and multiple update bugs fixed.

…opup, also removed dev code [ref #4949]

Will need to be bumped again after PR

…taverse into 4949-download-package-s3

Added the section "Downloading a Dataverse Package via URL". This should cover this feature for the User Guide.

Based on Phil's review, changed the docs to no longer wildly overestimate Wget's capabilities ;)

in the future this code may be used for non-s3 packages and so should be more agnostic.

coveralls · 2018-11-30T23:16:33Z

Coverage decreased (-0.03%) to 17.667% when pulling 5537175 on 4949-download-package-s3 into 15397f8 on develop.

mheppler · 2018-12-03T17:02:44Z

src/main/webapp/individualFileTable.xhtml

+    xmlns:o="http://omnifaces.org/ui"
+    xmlns:iqbs="http://xmlns.jcp.org/jsf/composite/iqbs">
+    <p:outputPanel>
+        <div id="fileInfoInclude-filesTable" class="col-sm-12 row">


I'd like to suggest a simple name change to this xhtml file, so that it will be grouped with the other file related files in the webapp directory. Something like "file-info-fragment" would do it.

pdurbin

I see that "a .rst page on how to set up dcm s3" is coming, which is awesome and I'll wait to approve this pull request until then.

Editing still needed

pdurbin

Overall, looks just fine but I noted a few changes we could make.

pdurbin · 2018-12-05T16:36:10Z

src/main/java/edu/harvard/iq/dataverse/S3PackageImporter.java

-                }
+        }
+
+        logger.log(Level.INFO, "Checksum value for the package in Dataset {0} is: {1}", 


Should this be FINE instead of INFO since it's the happy path?

pdurbin · 2018-12-05T16:39:46Z

src/main/java/edu/harvard/iq/dataverse/util/SystemConfig.java

         */
        RSYNC("rsal/rsync"),
-        NATIVE("NATIVE");
+        NATIVE("native/http");


Should we also update the tests in DataCaptureModuleUtilTest.java?

pdurbin · 2018-12-05T16:41:35Z

src/main/java/edu/harvard/iq/dataverse/util/SystemConfig.java

    }

+    public boolean isHTTPDownload()
+    {


We could move this curly to the line above, if we care to.

pdurbin · 2018-12-05T16:42:27Z

src/main/webapp/WEB-INF/web.xml

+        <param-name>javax.faces.PROJECT_STAGE</param-name>
+<!--        <param-value>Development</param-value>-->
+        <param-value>Production</param-value>
+    </context-param>


I'm very happy to see this in here because I always forget how to enable debugging in JSF.

pdurbin · 2018-12-05T16:44:06Z

src/main/webapp/individualFileTable.xhtml

+    xmlns:iqbs="http://xmlns.jcp.org/jsf/composite/iqbs">
+    <p:outputPanel>
+        <div id="fileInfoInclude-filesTable" class="col-sm-12 row">
+            <div class="pull-left col-file-thumb">


I just wanted to note that @mheppler made a comment above about file name conventions for xhtml files. From my perspective, we are not at all consistent. My approach would probably be to start by asserting a preference in the dev guide on the "Coding Style" page. Then we could work toward renaming existing files to comply with the naming standard.

pdurbin

Looks good as of c891407

includes the latex/pdf fix

matthew-a-dunlap and others added 25 commits November 1, 2018 16:24

Pull new dcm package file and zip checksum #4949

f41747d

Pull checksum from dcm uploaded .sha #5053

c8f80f0

Revert "Pull checksum from dcm uploaded .sha #5053"

4b55eba

This reverts commit c8f80f0.

Pull checksum from dcm uploaded .sha #4949

cbea0c6

Move file info to new fragment #4949

9284ba9

This move allows that code to be reused in the new package popup This also contains a bit of render logic around s3 package UI

Generate s3 identifier on import #4949

0cdc100

There was a bug in the previous story where the imported dcm s3 files did not have an S3 identifier.

Popup with s3 package download url #4949

f391e63

Merge branch 'develop' into 4949-download-package-s3

cc29b93

Launch s3Package popup from guestbook #4949

36b8648

dcm s3 file page start #4949

e7d7dcd

Share file entry code from replace and s3 #4949

756de2b

Wired up for all flows, and multiple update bugs fixed.

dcm s3 download code cleanup #4949

23196c0

Cleaned up UI of new popup for S3 download [ref #4949]

8827430

Added placeholder User Guide section for S3 download, and link from p…

1802ef0

…opup, also removed dev code [ref #4949]

Fixed header formatting issue in User Guide for S3 download [ref #4949]

51069be

dcms3 download friendly size display #4949

5da9584

Will need to be bumped again after PR

Merge branch '4949-download-package-s3' of https://github.com/IQSS/da…

75fabe9

…taverse into 4949-download-package-s3

Present correct script name dcms3 #4949

dbc13d5

dataset w. package not look for rsync script #4949

02a6622

Merge branch 'develop' into 4949-download-package-s3

930a77f

code cleanup and better bundle naming #4949

42e5977

Package Download User Guide Section [#4949]

ddf52d1

Added the section "Downloading a Dataverse Package via URL". This should cover this feature for the User Guide.

Update to docs - Wget isn't magic [#4949]

1a53227

Based on Phil's review, changed the docs to no longer wildly overestimate Wget's capabilities ;)

dcms3 download only with downloadMethod #4949

1db38db

remove s3 naming from package code #4949

a6d4f96

in the future this code may be used for non-s3 packages and so should be more agnostic.

mheppler reviewed Dec 3, 2018

View reviewed changes

pdurbin requested changes Dec 3, 2018

View reviewed changes

move&format dcm s3 docs

1938862

Editing still needed

pdurbin requested changes Dec 5, 2018

View reviewed changes

matthew-a-dunlap added 7 commits December 5, 2018 12:36

Switch file info frag name #4949

f109aaf

code cleanup and test fixing #4949

5e0a0fb

More doc cleanup #4949

ea76617

Merge branch 'develop' into 4949-download-package-s3

2f12cd8

Bump DCM version #4949

9a47958

Minor doc fix #4949

a170d78

Another minor doc change #4949

c891407

pdurbin approved these changes Dec 5, 2018

View reviewed changes

matthew-a-dunlap and others added 3 commits December 10, 2018 14:38

Fix refactor double extension #4949

4432cae

dcms3 code review docfixes #4949

5537175

includes the latex/pdf fix

dcms3 install fixes and enable restrict #4949

bf31f9e

kcondon merged commit 88c2494 into develop Dec 14, 2018

kcondon deleted the 4949-download-package-s3 branch December 14, 2018 17:11

pdurbin mentioned this pull request Dec 14, 2018

war file does not compile after pull request #5359 was merged #5403

Closed

pdurbin added this to the 4.10 - Additional Data Transfer Options milestone Dec 19, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

4949 download package s3#5359

4949 download package s3#5359
kcondon merged 36 commits intodevelopfrom
4949-download-package-s3

matthew-a-dunlap commented Nov 30, 2018

Uh oh!

coveralls commented Nov 30, 2018 •

edited

Loading

Uh oh!

mheppler Dec 3, 2018

Uh oh!

pdurbin left a comment

Uh oh!

pdurbin left a comment

Uh oh!

pdurbin Dec 5, 2018

Uh oh!

pdurbin Dec 5, 2018

Uh oh!

pdurbin Dec 5, 2018

Uh oh!

pdurbin Dec 5, 2018

Uh oh!

pdurbin Dec 5, 2018

Uh oh!

pdurbin left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Conversation

matthew-a-dunlap commented Nov 30, 2018

Related Issues

Pull Request Checklist

Uh oh!

coveralls commented Nov 30, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mheppler Dec 3, 2018

Choose a reason for hiding this comment

Uh oh!

pdurbin left a comment

Choose a reason for hiding this comment

Uh oh!

pdurbin left a comment

Choose a reason for hiding this comment

Uh oh!

pdurbin Dec 5, 2018

Choose a reason for hiding this comment

Uh oh!

pdurbin Dec 5, 2018

Choose a reason for hiding this comment

Uh oh!

pdurbin Dec 5, 2018

Choose a reason for hiding this comment

Uh oh!

pdurbin Dec 5, 2018

Choose a reason for hiding this comment

Uh oh!

pdurbin Dec 5, 2018

Choose a reason for hiding this comment

Uh oh!

pdurbin left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

coveralls commented Nov 30, 2018 •

edited

Loading