Fixed flaky tests in FlattenSpecParquetReaderTest.java with ObjectMapper

[CaseyPan]

Description

Fixed flaky tests in FlattenSpecParquetReaderTest.java with ObjectMapper

druid/extensions-core/parquet-extensions/src/test/java/org/apache/druid/data/input/parquet/FlattenSpecParquetReaderTest.java

Line 42 in ad32f84

public class FlattenSpecParquetReaderTest extends BaseParquetReaderTest

Test: testFlat1NoFlattenSpec, testFlat1Autodiscover, testFlat1Flatten, testFlat1FlattenSelectListItem, testNested1NoFlattenSpec, testNested1Autodiscover, testNested1Flatten, testNested1FlattenSelectListItem

Steps to Reproduce:

1. Clone the repo

   git clone git clone https://github.com/apache/druid.git
   

2. Install the package

   mvn install -pl extensions-core/parquet-extensions -am -DskipTests
   

3. Run the test in regular runs

   mvn -pl extensions-core/parquet-extensions test -Dtest=org.apache.druid.data.input.parquet.FlattenSpecParquetReaderTest#testname
   

4. The test failed by running the test with NonDex tool.

   mvn -pl extensions-core/parquet-extensions edu.illinois:nondex-maven-plugin:2.1.1:nondex -Dcheckstyle.skip -Dtest=org.apache.druid.data.input.parquet.FlattenSpecParquetReaderTest#testname
   

The following is an example error message from testFlat1Flatten

  [ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 2.839 s <<< FAILURE! - in org.apache.druid.data.input.parquet.FlattenSpecParquetReaderTest
  [ERROR] org.apache.druid.data.input.parquet.FlattenSpecParquetReaderTest.testFlat1Flatten - Time elapsed: 0.073 s <<< FAILURE!
  org.junit.ComparisonFailure: 
  expected:<{
    "ListDim" : [ "listDim1v1", "listDim1v2" ],
    "dim3" : 1,
    "dim2" : "d2v1",
    "dim1" : "d1v1",
    "metric1" : 1,
    "timestamp" : 1537229880023
  }> but was:<{
    "dim3" : 1,
    "timestamp" : 1537229880023,
    "dim2" : "d2v1",
    "ListDim" : [ "listDim1v1", "listDim1v2" ],
    "metric1" : 1,
    "dim1" : "d1v1"
  }>
  
  at org.junit.Assert.assertEquals(Assert.java:117)

Reason of flakiness:
These tests check the equality of two JSON strings. The data reading methods used createReader() in those tests do not retrieve data in a consistent order, which can possibly lead to different JSON representations of the same data during test runs.

Changes:

• Imported fasterxml.jackson package, and changed from using standard assertion method Assert to using JSONNode and ObjectMapper, which parse two JSON strings and convert them to a tree structure and each node in the trees are compared. Thus, the order of the elements in JSON strings is deterministic and the test passes.

Release note

Fixed: FlattenSpecParquetReaderTest.java tests no longer fail when running with the NonDex tool.

---

This PR has:

• been self-reviewed.
• added documentation for new or modified features or behaviors.
• a release note entry in the PR description.
• added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
• added or updated version, license, or notice information in licenses.yaml
• added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
• added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
• added integration tests.
• been tested in a test Druid cluster.

[krishnanand5]

In the release note, you can probably reword it to talk about just addressing the flakiness in the tests and not mention nondex. If space permits, you can probably add a failure stacktrace from one of the nondex runs. Apart from that, I feel the justification for leveraging additional dependencies is adequate.

[Rette66]

Could you attach more details to the cause and issue, the links to the code or snippets would also work.

[kavvya97]

Hello casey,
these are error logs.
[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 2.839 s <<< FAILURE! - in org.apache.druid.data.input.parquet.FlattenSpecParquetReaderTest
[ERROR] org.apache.druid.data.input.parquet.FlattenSpecParquetReaderTest.testFlat1 Flatten - Time elapsed: 0.073 s <<< FAILURE!
org.junit.ComparisonFailure:
expected:<{
"ListDim" : [ "listDim1v1", "listDim1v2" ],
"dim3" : 1,
"dim2" : "d2v1",
"dim1" : "d1v1",
"metric1" : 1,
"timestamp" : 1537229880023
}> but was:<{
"dim3" : 1,
"timestamp" : 1537229880023,
"dim2" : "d2v1",
"ListDim" : [ "listDim1v1", "listDim1v2" ],
"metric1" : 1,
"dim1" : "d1v1"
}>