[BUG] GH Pipelines does not indicate on failed test run

[mgaffigan]

Some PR's that have merged recently have failed tests, but the check still passed.

Presumably checks should not pass if tests do not pass.

[tonygermano]

You are correct that failed tests should block the PR, however, I think those SSL tests need to be updated and have always failed since before this project started. I briefly looked at them about a month ago, and I think they work under the assumption that java 8 doesn't have support for some protocols that have since been added. Soon we won't be compiling for java 8, anyway, and they will likely go away on their own.

[tonygermano]

Yeah, to verify, I just built mirth 4.5.2, and the same tests failed, so it wasn't anything we broke in oie.

test-run:
[jacoco:coverage] Enhancing junit with coverage
    [junit] TEST com.mirth.connect.util.MirthSSLUtilTest FAILED
    [junit] Tests FAILED

These are the two tests which failed

  <testcase classname="com.mirth.connect.util.MirthSSLUtilTest" name="testEnabledProtocols" time="0.007">
    <failure type="junit.framework.AssertionFailedError">junit.framework.AssertionFailedError: 
        at com.mirth.connect.util.MirthSSLUtilTest.testEnabledProtocols(MirthSSLUtilTest.java:37)
</failure>
  </testcase>
  <testcase classname="com.mirth.connect.util.MirthSSLUtilTest" name="testEnabledCipherSuites" time="0.001">
    <failure type="junit.framework.AssertionFailedError">junit.framework.AssertionFailedError: 
        at com.mirth.connect.util.MirthSSLUtilTest.testEnabledCipherSuites(MirthSSLUtilTest.java:54)
</failure>
  </testcase>

[tonygermano]

Also, not that you said otherwise, I believe the bug is in the ant script, and not in the github setup. Ant says the build was successful despite the failing tests.

[mgaffigan]

Fair. This can also be updated to use the separate build / test steps now available. That will give better visibility when running checks, and would permit checks on multiple JVM's if that is relevant.

Is there an issue open for the failing test?

I've always gotten the following test failures when running on windows. Should I open tickets for these? Or is there something wrong with my build environment?

[junit] TEST com.mirth.commons.encryption.test.KeyEncryptorTest FAILED
[junit] TEST com.mirth.connect.client.core.PropertiesConfigurationUtilTest FAILED
[junit] TEST com.mirth.connect.server.servlets.WebStartServletTest FAILED
[junit] TEST com.mirth.connect.server.util.MapUtilTest FAILED
[junit] TEST com.mirth.connect.util.MirthSSLUtilTest FAILED
[junit] Tests FAILED

[tonygermano]

Fair. This can also be updated to use the separate build / test steps now available. That will give better visibility when running checks, and would permit checks on multiple JVM's if that is relevant.

Sounds good to me. That's how I was expecting it to work once we had gradle up and running.

Is there an issue open for the failing test?

I don't think I opened an issue for it. I was looking at it really late one night, and then I forgot about it until you brought it up again. I think I had reached the conclusion that we'd soon be building with a current java version and targeting 17 and the failure would go away.

I've always gotten the following test failures when running on windows. Should I open tickets for these? Or is there something wrong with my build environment?

[junit] TEST com.mirth.commons.encryption.test.KeyEncryptorTest FAILED
[junit] TEST com.mirth.connect.client.core.PropertiesConfigurationUtilTest FAILED
[junit] TEST com.mirth.connect.server.servlets.WebStartServletTest FAILED
[junit] TEST com.mirth.connect.server.util.MapUtilTest FAILED
[junit] TEST com.mirth.connect.util.MirthSSLUtilTest FAILED
[junit] Tests FAILED

Not sure about the other failures. I develop locally in linux and have only seen the MirthSSLUtilTest failures (admittedly, I usually don't scan the results as most of the changes I've been looking at recently are non-code changes.) I can try building on windows in the next few days to see if I can reproduce.

[mgaffigan]

Maybe a JVM option issue? Some failures:

server\test\com\mirth\commons\encryption\test\KeyEncryptorTest.java:

Charset defaultCharset = Charset.defaultCharset();
assertFalse(defaultCharset.name().equals(Charset.forName("windows-1252").name()));

This one seems like a build process issue maybe?

com.mirth.connect.server.util.MapUtilTest:

java.sql.SQLException: Failed to create database 'MapUtilTestDB', see the next exception for details.
	at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)
        ...
	at java.sql.DriverManager.getConnection(DriverManager.java:270)
	at com.mirth.connect.server.userutil.DatabaseConnection.<init>(DatabaseConnection.java:49)
	at com.mirth.connect.server.userutil.DatabaseConnectionFactory.createDatabaseConnection(DatabaseConnectionFactory.java:85)
	at com.mirth.connect.server.util.MapUtilTest.testNonSerializableValue2(MapUtilTest.java:83)
Caused by: java.sql.SQLException: Failed to create database 'MapUtilTestDB', see the next exception for details.
        ...
Caused by: java.sql.SQLException: The database directory 'C:\dev\gitroot\oie\server\MapUtilTestDB' exists. However, it 
does not contain the expected 'service.properties' file. Perhaps Derby was brought down in the middle of creating this 
database. You may want to delete this directory and try creating the database again.
        ...
Caused by: ERROR XBM0A: The database directory 'C:\dev\gitroot\oie\server\MapUtilTestDB' exists. However, it does 
not contain the expected 'service.properties' file. Perhaps Derby was brought down in the middle of creating this 
database. You may want to delete this directory and try creating the database again.
	at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
	at org.apache.derby.impl.services.monitor.StorageFactoryService.vetService(Unknown Source)
	at org.apache.derby.impl.services.monitor.StorageFactoryService.access$900(Unknown Source)
	at org.apache.derby.impl.services.monitor.StorageFactoryService$10.run(Unknown Source)
	at java.security.AccessController.doPrivileged(Native Method)
	at org.apache.derby.impl.services.monitor.StorageFactoryService.createServiceRoot(Unknown Source)
	at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
	at org.apache.derby.impl.services.monitor.BaseMonitor.createPersistentService(Unknown Source)
	at org.apache.derby.iapi.services.monitor.Monitor.createPersistentService(Unknown Source)

Not sure about the others. I'll take a look later and open separate tickets. I don't want to muddy this thread.