Files: File download for R, CSV, or original file format

[eaquigley]

---

Author Name: Kevin Condon (@kcondon)
Original Redmine Issue: 3566, https://redmine.hmdc.harvard.edu/issues/3566
Original Date: 2014-02-25
Original Assignee: Kevin Condon

---

From 2/21/14 Pilot Test Meeting Notes

File download for: R, CSV, or original file format

[eaquigley]

---

Original Redmine Comment
Author Name: Leonid Andreev (@landreev)
Original Date: 2014-02-27T20:08:44Z

---

Will be done by tomorrow - minus R format (will be shaded).
The way the application is talking to R is being rebuilt (switching to apacheR from rserve), so I can't generate R data frames yet.

[eaquigley]

---

Original Redmine Comment
Author Name: Leonid Andreev (@landreev)
Original Date: 2014-02-28T17:34:18Z

---

Tabular ("subsettable") data files, once ingested, should be showing an expanding download menu.
The menu item for R data format is disabled/shaded (per my previous) comment.
Download and download-saved-original should be working.

[eaquigley]

---

Original Redmine Comment
Author Name: Elda Sotiri (@esotiri)
Original Date: 2014-03-05T21:13:38Z

---

getting exception adding a stata file, here is part of sys log.

[2014-03-05T16:10:35.875-0500] [glassfish 4.0] [WARNING] [] [org.eclipse.persistence.session.file:/usr/local/glassfish4/glassfish/domains/domain1/applications/dataverse-4.0/WEB-INF/classes/_VDCNet-ejbPU] [tid: _ThreadID=20 _ThreadName=http-listener-1(3)] [timeMillis: 1394053835875] [levelValue: 900] [[

Local Exception Stack:
Exception [EclipseLink-4002](Eclipse Persistence Services - 2.5.0.v20130507-3faac2b): org.eclipse.persistence.exceptions.DatabaseException
Internal Exception: org.postgresql.util.PSQLException: ERROR: permission denied for sequence filesystemname_seq
Error Code: 0
Call: select nextval('filesystemname_seq')
Query: DataReadQuery(sql="select nextval('filesystemname_seq')")
at org.eclipse.persistence.exceptions.DatabaseException.sqlException(DatabaseException.java:340)
at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.processExceptionForCommError(DatabaseAccessor.java:1605)
at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.basicExecuteCall(DatabaseAccessor.java:671)
at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.executeCall(DatabaseAccessor.java:558)
at org.eclipse.persistence.internal.sessions.AbstractSession.basicExecuteCall(AbstractSession.java:1995)
at org.eclipse.persistence.sessions.server.ClientSession.executeCall(ClientSession.java:296)
at org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.executeCall(DatasourceCallQueryMechanism.java:242)
at org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.executeCall(DatasourceCallQueryMechanism.java:228)
at org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.executeSelectCall(DatasourceCallQueryMechanism.java:299)
at org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.executeSelect(DatasourceCallQueryMechanism.java:281)
at org.eclipse.persistence.queries.DataReadQuery.executeNonCursor(DataReadQuery.java:197)
at org.eclipse.persistence.queries.DataReadQuery.executeDatabaseQuery(DataReadQuery.java:152)
at org.eclipse.persistence.queries.DatabaseQuery.execute(DatabaseQuery.java:899)
at org.eclipse.persistence.queries.DataReadQuery.execute(DataReadQuery.java:137)
at org.eclipse.persistence.queries.DatabaseQuery.executeInUnitOfWork(DatabaseQuery.java:798)
at org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.internalExecuteQuery(UnitOfWorkImpl.java:2894)
at org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1797)
at org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1779)
at org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1744)
at org.eclipse.persistence.internal.jpa.QueryImpl.executeReadQuery(QueryImpl.java:258)
at org.eclipse.persistence.internal.jpa.QueryImpl.getSingleResult(QueryImpl.java:516)
at org.eclipse.persistence.internal.jpa.EJBQueryImpl.getSingleResult(EJBQueryImpl.java:400)
at edu.harvard.iq.dataverse.DatasetServiceBean.generateFileSystemName(DatasetServiceBean.java:82)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.glassfish.ejb.security.application.EJBSecurityManager.runMethod(EJBSecurityManager.java:1081)
at org.glassfish.ejb.security.application.EJBSecurityManager.invoke(EJBSecurityManager.java:1153)
at com.sun.ejb.containers.BaseContainer.invokeBeanMethod(BaseContainer.java:4695)
at com.sun.ejb.EjbInvocation.invokeBeanMethod(EjbInvocation.java:630)
at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:822)
at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:582)
at org.jboss.weld.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:46)
at sun.reflect.GeneratedMethodAccessor85.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:822)
at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:582)
at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.doCall(SystemInterceptorProxy.java:163)
at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.aroundInvoke(SystemInterceptorProxy.java:140)
at sun.reflect.GeneratedMethodAccessor93.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:822)
at com.sun.ejb.containers.interceptors.InterceptorManager.intercept(InterceptorManager.java:369)
at com.sun.ejb.containers.BaseContainer.intercept(BaseContainer.java:4667)
at com.sun.ejb.containers.BaseContainer.intercept(BaseContainer.java:4655)
at com.sun.ejb.containers.EJBLocalObjectInvocationHandler.invoke(EJBLocalObjectInvocationHandler.java:212)
at com.sun.ejb.containers.EJBLocalObjectInvocationHandlerDelegate.invoke(EJBLocalObjectInvocationHandlerDelegate.java:88)
at com.sun.proxy.$Proxy188.generateFileSystemName(Unknown Source)
at edu.harvard.iq.dataverse.__EJB31_Generated__DatasetServiceBean__Intf____Bean.generateFileSystemName(Unknown Source)
at edu.harvard.iq.dataverse.DatasetPage.save(DatasetPage.java:403)
at edu.harvard.iq.dataverse.DatasetPage$Proxy$_$$_WeldClientProxy.save(Unknown Source)
at sun.reflect.GeneratedMethodAccessor505.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at javax.el.ELUtil.invokeMethod(ELUtil.java:326)
at javax.el.BeanELResolver.invoke(BeanELResolver.java:536)
at javax.el.CompositeELResolver.invoke(CompositeELResolver.java:256)
at com.sun.el.parser.AstValue.invoke(AstValue.java:269)
at com.sun.el.MethodExpressionImpl.invoke(MethodExpressionImpl.java:304)
at org.jboss.weld.util.el.ForwardingMethodExpression.invoke(ForwardingMethodExpression.java:40)
at org.jboss.weld.el.WeldMethodExpression.invoke(WeldMethodExpression.java:50)
at com.sun.faces.facelets.el.TagMethodExpression.invoke(TagMethodExpression.java:105)
at javax.faces.component.MethodBindingMethodExpressionAdapter.invoke(MethodBindingMethodExpressionAdapter.java:87)
at com.sun.faces.application.ActionListenerImpl.processAction(ActionListenerImpl.java:102)
at javax.faces.component.UICommand.broadcast(UICommand.java:315)
at javax.faces.component.UIViewRoot.broadcastEvents(UIViewRoot.java:790)
at javax.faces.component.UIViewRoot.processApplication(UIViewRoot.java:1282)
at com.sun.faces.lifecycle.InvokeApplicationPhase.execute(InvokeApplicationPhase.java:81)
at com.sun.faces.lifecycle.Phase.doPhase(Phase.java:101)
at com.sun.faces.lifecycle.LifecycleImpl.execute(LifecycleImpl.java:198)
at javax.faces.webapp.FacesServlet.service(FacesServlet.java:646)
at org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1682)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:318)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:160)
at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:734)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:673)
at com.sun.enterprise.web.WebPipeline.invoke(WebPipeline.java:99)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:174)
at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:734)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:673)
at org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:354)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:260)
at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:188)
at org.glassfish.grizzly.http.server.HttpHandler.runService(HttpHandler.java:196)
at org.glassfish.grizzly.http.server.HttpHandler.doHandle(HttpHandler.java:173)
at org.glassfish.grizzly.http.server.HttpServerFilter.handleRead(HttpServerFilter.java:189)
at org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:119)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:288)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:206)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:136)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:114)
at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:77)
at org.glassfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:838)
at org.glassfish.grizzly.strategies.AbstractIOStrategy.fireIOEvent(AbstractIOStrategy.java:113)
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.run0(WorkerThreadIOStrategy.java:115)
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.access$100(WorkerThreadIOStrategy.java:55)
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy$WorkerThreadRunnable.run(WorkerThreadIOStrategy.java:135)
at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:564)
at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:544)
at java.lang.Thread.run(Thread.java:744)
Caused by: org.postgresql.util.PSQLException: ERROR: permission denied for sequence filesystemname_seq
at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:1592)
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1327)
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:192)
at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:451)
at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:350)
at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:254)
at sun.reflect.GeneratedMethodAccessor67.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.postgresql.ds.jdbc23.AbstractJdbc23PooledConnection$StatementHandler.invoke(AbstractJdbc23PooledConnection.java:477)
at com.sun.proxy.$Proxy290.executeQuery(Unknown Source)
at com.sun.gjc.spi.jdbc40.PreparedStatementWrapper40.executeQuery(PreparedStatementWrapper40.java:642)
at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.executeSelect(DatabaseAccessor.java:1002)
at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.basicExecuteCall(DatabaseAccessor.java:641)
... 102 more
]]

[2014-03-05T16:10:35.885-0500] [glassfish 4.0] [WARNING] [ejb.system_exception] [javax.enterprise.system.container.ejb.com.sun.ejb.containers] [tid: _ThreadID=20 _ThreadName=http-listener-1(3)] [timeMillis: 1394053835885] [levelValue: 900] [[
EJB5184:A system exception occurred during an invocation on EJB DatasetServiceBean, method: public void edu.harvard.iq.dataverse.DatasetServiceBean.generateFileSystemName(edu.harvard.iq.dataverse.DataFile)]]

[2014-03-05T16:10:35.886-0500] [glassfish 4.0] [WARNING] [] [javax.enterprise.system.container.ejb.com.sun.ejb.containers] [tid: _ThreadID=20 _ThreadName=http-listener-1(3)] [timeMillis: 1394053835886] [levelValue: 900] [[

javax.ejb.EJBException
at com.sun.ejb.containers.EJBContainerTransactionManager.processSystemException(EJBContainerTransactionManager.java:748)
at com.sun.ejb.containers.EJBContainerTransactionManager.completeNewTx(EJBContainerTransactionManager.java:698)
at com.sun.ejb.containers.EJBContainerTransactionManager.postInvokeTx(EJBContainerTransactionManager.java:503)
at com.sun.ejb.containers.BaseContainer.postInvokeTx(BaseContainer.java:4475)
at com.sun.ejb.containers.BaseContainer.postInvoke(BaseContainer.java:2009)

[eaquigley]

---

Original Redmine Comment
Author Name: Leonid Andreev (@landreev)
Original Date: 2014-03-05T21:22:47Z

---

Elda,
Note that this is not an Ingest ticket - this is a "download as ..." ticket.
But yes, I need to own this ticket for now, since I need to enable downloading the datafile in R format (currently disabled).

[eaquigley]

---

Original Redmine Comment
Author Name: Leonid Andreev (@landreev)
Original Date: 2014-03-17T03:47:36Z

---

Tabular data files (aka "subsettable" files) now have a download menu with the 4 options:

• as tab-delimited (NOT as CSV, as originally specified! we're staying with Tab, at least for now)
• as RData
• the saved original (with the name of the original format in parenthesis)
• "variable metadata" (this is an addition to the original specifications; returns a DDI XML fragment with the "stdyFile" and "dataDscr" sections with the metadata specific to this file)

[eaquigley]

---

Original Redmine Comment
Author Name: Kevin Condon (@kcondon)
Original Date: 2014-04-02T18:38:10Z

---

Tested on 4/2

Basic download options are present and work.

Will do more comprehensive testing later when all file functionality is delivered.

Closing ticket