Skip to content

Comments

7418 datasourcedefinition#7422

Merged
kcondon merged 24 commits intoIQSS:developfrom
poikilotherm:7418-datasourcedefinition
Dec 4, 2020
Merged

7418 datasourcedefinition#7422
kcondon merged 24 commits intoIQSS:developfrom
poikilotherm:7418-datasourcedefinition

Conversation

@poikilotherm
Copy link
Contributor

What this PR does / why we need it:
This will move the database connection configuration from the application server down to the application.
This greatly simplifies how to configure the connection details by using MicroProfile Config API.

Which issue(s) this PR closes:

Closes #7418
Requires #7416
Requires #7419
Requires #7420

Special notes for your reviewer:
None.

Suggestions on how to test this:
Be sure to apply the PRs mentioned above first. Follow release notes instructions. Do usual testing.

Does this PR introduce a user interface change? If mockups are available, please link/include them here:
Nope.

Is there a release notes update needed for this change?:
🔋 included

Additional documentation:
None.

@poikilotherm poikilotherm added Type: Feature a feature request Feature: Installer Feature: Installation Guide User Role: Sysadmin Installs, upgrades, and configures the system, connects via ssh labels Nov 18, 2020
@coveralls
Copy link

coveralls commented Nov 18, 2020

Coverage Status

Coverage decreased (-0.0005%) to 19.397% when pulling 7e3498e on poikilotherm:7418-datasourcedefinition into c1efd01 on IQSS:develop.

@poikilotherm
Copy link
Contributor Author

poikilotherm commented Nov 24, 2020

Now that #7419, #7420 and #7416 have been merged, shall we move this to Code Review or QA @pdurbin ?

@pdurbin
Copy link
Member

pdurbin commented Nov 24, 2020

@poikilotherm code review sounds fine after you merge the latest from develop. Thanks.

@poikilotherm
Copy link
Contributor Author

@poikilotherm code review sounds fine after you merge the latest from develop. Thanks.

It looked like GitHub says it doesn't conflict. Should I update nonetheless?

@pdurbin pdurbin self-assigned this Nov 24, 2020
@pdurbin
Copy link
Member

pdurbin commented Nov 24, 2020

@poikilotherm I went ahead and merged the latest from develop into your branch in 813e0fb and put it in code review.

src/main/resources/META-INF/microprofile-config.properties is important
for developers to know about but sysadmins following the installation
guide need not concern themselves with it.

Other tweaks in this commit are cosmetic. I de-emphasized the
environment variables by putting them in the bottom.
Copy link
Member

@pdurbin pdurbin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested this pull request on a fresh dev installation but I have not tested the upgrade scenarios presented in the release note so they should be tested. I also didn't test setting the new environment variables.

@donsizemore and I are aware that Jenkins is failing on this branch but we have a lead on how to fix it and it shouldn't require any changes to this pull request. In short we need to make sure that the installation scripts in this branch are used on the instance spun up by Jenkins.

public class StartupFlywayMigrator {

@Resource(lookup = "jdbc/VDCNetDS")
@Resource(lookup = "java:app/jdbc/dataverse")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In 9dfe7b8 it is explained why this changed:

The JNDI names have been changed to be conform with Java EE 7.
See https://github.com/javaee-samples/javaee7-samples/tree/master/jpa/datasourcedefinition-applicationxml-pu
and others for working examples. (Staying with the old name was not successfull.)

<include>**/*.xml</include>
<include>**/firstNames/*.*</include>
<include>**/*.xsl</include>
<include>**/*.properties</include>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was added so the following file would be included:

src/main/resources/META-INF/microprofile-config.properties

@pdurbin pdurbin removed their assignment Nov 25, 2020
Co-authored-by: Philip Durbin <philip_durbin@harvard.edu>
@pdurbin
Copy link
Member

pdurbin commented Dec 1, 2020

@poikilotherm we're seeing a number of failures on this branch (screenshot below) but not on develop. Any idea what's going on? We're happy to re-run the tests on your branch, of course, and help troubleshoot.

Screen Shot 2020-12-01 at 11 03 30 AM

@kcondon
Copy link
Contributor

kcondon commented Dec 1, 2020

@poikilotherm I was able to deploy the app, create user and dataverse but ran into an exception when creating a dataset, perhaps due to ingesting a file. I'll try to refine it some more. Yes, was able to individually create a dataset, then upload text and image files. When I tried saving a tabular Stata file, I saw the error.

[2020-12-01T16:49:53.994+0000] [Payara 5.2020.6] [INFO] [] [edu.harvard.iq.dataverse.ingest.IngestServiceBean] [tid: _ThreadID=89 _ThreadName=http-thread-pool::jk-connector(2)] [timeMillis: 1606841393994] [levelValue: 800] [[
Ingest of 1 tabular data file(s) is in progress.]]

[2020-12-01T16:49:54.082+0000] [Payara 5.2020.6] [WARNING] [] [javax.enterprise.system.core.transaction.com.sun.jts.CosTransactions] [tid: _ThreadID=89 _ThreadName=http-thread-pool::jk-connector(2)] [timeMillis: 1606841394082] [levelValue: 900] [[
JTS5031: Exception [java.lang.RuntimeException: org.postgresql.xa.PGXAException: Error preparing transaction. prepare xid=(Available at FINE log level)] on Resource [prepare] operation.]]

[2020-12-01T16:49:54.085+0000] [Payara 5.2020.6] [WARNING] [] [javax.enterprise.system.core.transaction.com.sun.jts.jtsxa] [tid: _ThreadID=89 _ThreadName=http-thread-pool::jk-connector(2)] [timeMillis: 1606841394085] [levelValue: 900] [[
JTS5068: Unexpected error occurred in rollback
org.postgresql.xa.PGXAException: Error rolling back prepared transaction. rollback xid=(Available at FINE log level), preparedXid=(Available at FINE log level), currentXid={2}
at org.postgresql.xa.PGXAConnection.rollback(PGXAConnection.java:485)
at com.sun.gjc.spi.XAResourceImpl.rollback(XAResourceImpl.java:204)
at com.sun.jts.jta.TransactionState._rollback(TransactionState.java:202)
at com.sun.jts.jta.TransactionState.rollback(TransactionState.java:180)
at com.sun.jts.jtsxa.OTSResourceImpl.rollback(OTSResourceImpl.java:333)
at com.sun.jts.CosTransactions.RegisteredResources.distributeRollback(RegisteredResources.java:1040)
at com.sun.jts.CosTransactions.TopCoordinator.rollback(TopCoordinator.java:2341)
at com.sun.jts.CosTransactions.CoordinatorTerm.commit(CoordinatorTerm.java:391)
at com.sun.jts.CosTransactions.TerminatorImpl.commit(TerminatorImpl.java:231)
at com.sun.jts.CosTransactions.CurrentImpl.commit(CurrentImpl.java:622)
at com.sun.jts.jta.TransactionManagerImpl.commit(TransactionManagerImpl.java:331)
at com.sun.enterprise.transaction.jts.JavaEETransactionManagerJTSDelegate.commitDistributedTransaction(JavaEETransactionManagerJTSDelegate.java:183)
at com.sun.enterprise.transaction.JavaEETransactionManagerSimplified.commit(JavaEETransactionManagerSimplified.java:965)
at com.sun.ejb.containers.EJBContainerTransactionManager.completeNewTx(EJBContainerTransactionManager.java:666)
at com.sun.ejb.containers.EJBContainerTransactionManager.postInvokeTx(EJBContainerTransactionManager.java:482)
at com.sun.ejb.containers.BaseContainer.postInvokeTx(BaseContainer.java:4583)
at com.sun.ejb.containers.BaseContainer.postInvoke(BaseContainer.java:2116)
at com.sun.ejb.containers.BaseContainer.postInvoke(BaseContainer.java:2086)
at com.sun.ejb.containers.EJBLocalObjectInvocationHandler.invoke(EJBLocalObjectInvocationHandler.java:220)
at com.sun.ejb.containers.EJBLocalObjectInvocationHandlerDelegate.invoke(EJBLocalObjectInvocationHandlerDelegate.java:90)
at com.sun.proxy.$Proxy318.startIngestJobsForDataset(Unknown Source)
at edu.harvard.iq.dataverse.ingest.EJB31_Generated__IngestServiceBean__Intf____Bean.startIngestJobsForDataset(Unknown Source)
at edu.harvard.iq.dataverse.DatasetPage.save(DatasetPage.java:3701)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.sun.el.util.ReflectionUtil.invokeMethod(ReflectionUtil.java:163)
at com.sun.el.parser.AstValue.invoke(AstValue.java:261)
at com.sun.el.MethodExpressionImpl.invoke(MethodExpressionImpl.java:237)
at org.jboss.weld.module.web.util.el.ForwardingMethodExpression.invoke(ForwardingMethodExpression.java:40)
at org.jboss.weld.module.web.el.WeldMethodExpression.invoke(WeldMethodExpression.java:50)
at com.sun.faces.facelets.el.TagMethodExpression.invoke(TagMethodExpression.java:65)
at com.sun.faces.application.MethodBindingMethodExpressionAdapter.invoke(MethodBindingMethodExpressionAdapter.java:66)
at com.sun.faces.application.ActionListenerImpl.getNavigationOutcome(ActionListenerImpl.java:82)
at com.sun.faces.application.ActionListenerImpl.processAction(ActionListenerImpl.java:71)
at javax.faces.component.UICommand.broadcast(UICommand.java:222)
at javax.faces.component.UIViewRoot.broadcastEvents(UIViewRoot.java:847)
at javax.faces.component.UIViewRoot.processApplication(UIViewRoot.java:1396)
at com.sun.faces.lifecycle.InvokeApplicationPhase.execute(InvokeApplicationPhase.java:58)
at com.sun.faces.lifecycle.Phase.doPhase(Phase.java:76)
at com.sun.faces.lifecycle.LifecycleImpl.execute(LifecycleImpl.java:177)
at javax.faces.webapp.FacesServlet.executeLifecyle(FacesServlet.java:707)
at javax.faces.webapp.FacesServlet.service(FacesServlet.java:451)
at org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1636)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:331)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:211)
at org.glassfish.tyrus.servlet.TyrusServletFilter.doFilter(TyrusServletFilter.java:282)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:253)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:211)
at org.ocpsoft.rewrite.servlet.RewriteFilter.doFilter(RewriteFilter.java:226)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:253)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:211)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:257)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161)
at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:757)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:577)
at com.sun.enterprise.web.WebPipeline.invoke(WebPipeline.java:99)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:158)
at org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:371)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:238)
at com.sun.enterprise.v3.services.impl.ContainerMapper$HttpHandlerCallable.call(ContainerMapper.java:520)
at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:217)
at org.glassfish.grizzly.http.server.HttpHandler.runService(HttpHandler.java:182)
at org.glassfish.grizzly.http.server.HttpHandler.doHandle(HttpHandler.java:156)
at org.glassfish.grizzly.http.server.HttpServerFilter.handleRead(HttpServerFilter.java:218)
at org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:95)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:260)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:177)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:109)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:88)
at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:53)
at org.glassfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:524)
at org.glassfish.grizzly.strategies.AbstractIOStrategy.fireIOEvent(AbstractIOStrategy.java:89)
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.run0(WorkerThreadIOStrategy.java:94)
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.access$100(WorkerThreadIOStrategy.java:33)
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy$WorkerThreadRunnable.run(WorkerThreadIOStrategy.java:114)
at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:569)
at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:549)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.postgresql.util.PSQLException: ERROR: prepared transaction with identifier "4871251_fwAAAM/wMx9pcC0xNzItMzEtMjQtMzcuZWMyLmludGVybmFsLHNlcnZlcixQMTAw_aXAtMTcyLTMxLTI0LTM3LmVjMi5pbnRlcm5hbCxzZXJ2ZXIsUDEwMCwB" does not exist
at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2553)
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2285)
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:323)
at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:473)
at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:393)
at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:322)
at org.postgresql.jdbc.PgStatement.executeCachedSql(PgStatement.java:308)
at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:284)
at org.postgresql.jdbc.PgStatement.executeUpdate(PgStatement.java:258)
at org.postgresql.xa.PGXAConnection.rollback(PGXAConnection.java:462)
... 79 more
]]

@poikilotherm
Copy link
Contributor Author

As for the check before using, I hear what he is saying but we did it for a specific reason -we were having real db connection pool issues and this allowed it to function more normally. I suppose the underlying problem may have gone away in the intervening time but I know he frequently requests that we are systematic about our conclusions.

Turns out I was a loudmouth. Payara does not support variables substitution in all property values, so this would need to be hardcoded as a workaround if absolutely necessary.

I opened payara/Payara#5024 to enhance Payara and come back to this in a later enhancement to our beloved codebase.

@kcondon @landreev @pdurbin could you folks please decide if a workaround is necessary? A quick patch is also possible, example in the code 😉

@kcondon kcondon removed their assignment Dec 2, 2020
Copy link
Member

@pdurbin pdurbin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm suggesting that we remove MPCONFIG stuff that doesn't work.

@pdurbin
Copy link
Member

pdurbin commented Dec 2, 2020

@poikilotherm yes, I think it would be nice to understand the workaround for configuring things that we used to. I just left a review suggesting a concrete example:

properties = {
    "fish.payara.is-connection-validation-required=true",
    "fish.payara.validation-table-name=setting"
}

Locally I could try changing these settings, recompiling, and deploying. But how do I know they took effect? Is there a list-jvm-options or something? If it's easier to detect changes to other settings I'd be happy to use those instead.

Again, see also the review I just left. The heart of it is that I think it's confusing to see MPCONFIG stuff in places where it doesn't work yet. Thanks.

Update: I just played around with adding the properties above. In practice, it looks like this:

-    maxIdleTime = 300)
+    maxIdleTime = 300,
+    properties = {
+        "fish.payara.is-connection-validation-required=true",
+        "fish.payara.validation-table-name=setting"
+    })

Then I compiled and deployed the app but I don't know how to tell if these settings above are active. I searched a bit for an asadmin command and finally poked around in the Payara admin GUI (screenshot below) especially under the application (dataverse-5.2) and the JDBC stuff but I couldn't find anything. My understanding is that the properties above should show up somewhere under the application itself rather than at the app server level.

Screen Shot 2020-12-02 at 2 24 30 PM

@pdurbin pdurbin assigned poikilotherm and pdurbin and unassigned poikilotherm Dec 2, 2020
@pdurbin
Copy link
Member

pdurbin commented Dec 3, 2020

I had a nice chat with @poikilotherm this morning and I'm going to try testing properties a bit more and make some of the doc changes I suggested.

- Show a simple properties example that's easily verifiable by inspecting
server.log (JDBC call debugging).
- Remove MPCONFIG examples that don't work yet (depends on work in
Payara).
@pdurbin
Copy link
Member

pdurbin commented Dec 3, 2020

To test properties I set fish.payara.log-jdbc-calls=true and it definitely worked. Just loading the home page generated 4035 lines in server.log. At this point I'm satisfied that if an installation of Dataverse really wants to set one of the many properties listed at https://docs.payara.fish/community/docs/5.2020.6/documentation/payara-server/jdbc/advanced-connection-pool-properties.html#full-list-of-properties they can edit DataSourceProducer.java, recompile, and it should work.

It would better to avoid having to compile, which is what payara/Payara#5024 is about.

@kcondon and I had a nice chat and talked about how if things really go south, we can always put out a point release that more or less reverts this pull request, putting us back on the older configuration we've used for years. Also, an individual installation can recompile if need be, as mentioned above. Hopefully we simply won't see any problems. 😄

In addition to this little bit of testing and a lot of talking, I made a few tweaks to the javadoc and release notes. I'm sending this to QA.

@pdurbin pdurbin removed their assignment Dec 3, 2020
@kcondon kcondon self-assigned this Dec 4, 2020
@kcondon kcondon merged commit ee3b4f4 into IQSS:develop Dec 4, 2020
pdurbin added a commit that referenced this pull request Dec 4, 2020
@poikilotherm poikilotherm deleted the 7418-datasourcedefinition branch December 4, 2020 23:34
@poikilotherm
Copy link
Contributor Author

Wooohoooo we're there finally!

🌟😎🎉🥳😎🎉🥳🌟

Thank you @kcondon , @pdurbin and @landreev for your support with this.

kcondon added a commit that referenced this pull request Dec 7, 2020
db_password_alias changed to dataverse.db.password #7418 #7422
@djbrooke djbrooke added this to the 5.3 milestone Dec 7, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Feature: Installation Guide Feature: Installer Type: Feature a feature request User Role: Sysadmin Installs, upgrades, and configures the system, connects via ssh

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Move database configuration into application

5 participants