Skip to content

Refactor metrics tests for reuse#331

Merged
rdblue merged 10 commits intoapache:masterfrom
edgarRd:metrics-tests
Aug 1, 2019
Merged

Refactor metrics tests for reuse#331
rdblue merged 10 commits intoapache:masterfrom
edgarRd:metrics-tests

Conversation

@edgarRd
Copy link
Copy Markdown
Contributor

@edgarRd edgarRd commented Jul 30, 2019

This PR includes the following changes:

  • Refactor Metrics tests for reuse.
  • Relocate truncation tests that are not parquet specific.

@rdblue @vgankidi

Comment thread build.gradle Outdated
@edgarRd edgarRd changed the title Refactor tests for metrics for reuse Refactor metrics tests for reuse Jul 31, 2019
Comment thread core/src/test/java/org/apache/iceberg/TestMetricsTruncation.java Outdated
@rdblue
Copy link
Copy Markdown
Contributor

rdblue commented Jul 31, 2019

@edgarRd, nice work! Just one thing to fix and I'll merge this.

@edgarRd
Copy link
Copy Markdown
Contributor Author

edgarRd commented Jul 31, 2019

CI is failing, it seems like there are some non-deterministic errors in the test suite already in master:

org.apache.iceberg.spark.source.TestIcebergSourceHiveTables > testHiveFilesTable FAILED
    java.lang.RuntimeException at TestIcebergSourceHiveTables.java:196
        Caused by: org.apache.thrift.transport.TTransportException at TestIcebergSourceHiveTables.java:196
            Caused by: java.net.SocketException at TestIcebergSourceHiveTables.java:196

org.apache.iceberg.spark.source.TestIcebergSourceHiveTables > testHiveFilesUnpartitionedTable FAILED
    java.lang.RuntimeException at TestIcebergSourceHiveTables.java:233
        Caused by: org.apache.thrift.transport.TTransportException at TestIcebergSourceHiveTables.java:233
            Caused by: java.net.SocketException at TestIcebergSourceHiveTables.java:233

org.apache.iceberg.spark.source.TestIcebergSourceHiveTables > testHiveManifestsTable FAILED
    java.lang.RuntimeException at TestIcebergSourceHiveTables.java:438
        Caused by: org.apache.thrift.transport.TTransportException at TestIcebergSourceHiveTables.java:438
            Caused by: java.net.SocketException at TestIcebergSourceHiveTables.java:438

org.apache.iceberg.spark.source.TestIcebergSourceHiveTables > testHiveHistoryTable FAILED
    java.lang.RuntimeException at TestIcebergSourceHiveTables.java:285
        Caused by: org.apache.thrift.transport.TTransportException at TestIcebergSourceHiveTables.java:285
            Caused by: java.net.SocketException at TestIcebergSourceHiveTables.java:285

org.apache.iceberg.spark.source.TestIcebergSourceHiveTables > testHiveEntriesTable FAILED
    java.lang.RuntimeException at TestIcebergSourceHiveTables.java:144
        Caused by: org.apache.thrift.transport.TTransportException at TestIcebergSourceHiveTables.java:144
            Caused by: java.net.SocketException at TestIcebergSourceHiveTables.java:144

@rdblue
Copy link
Copy Markdown
Contributor

rdblue commented Jul 31, 2019

I restarted CI. I'll have to take a look at the flaky test, too.

@rdblue rdblue merged commit 26765c8 into apache:master Aug 1, 2019
@rdblue
Copy link
Copy Markdown
Contributor

rdblue commented Aug 1, 2019

Merged. Thanks for fixing this, @edgarRd!

@edgarRd
Copy link
Copy Markdown
Contributor Author

edgarRd commented Aug 1, 2019

Great, thanks!

@edgarRd edgarRd deleted the metrics-tests branch August 1, 2019 00:24
danielcweeks pushed a commit that referenced this pull request Aug 1, 2019
* Add argument validation to HadoopTables#create (#298)

* Install source JAR when running install target (#310)

* Add projectStrict for Dates and Timestamps (#283)

* Correctly publish artifacts on JitPack (#321)

The Gradle install target produces invalid POM files that are missing
the dependencyManagement section and versions for some dependencies.
Instead, we directly tell JitPack to run the correct Gradle target.

* Add build info to README.md (#304)

* Convert Iceberg time type to Hive string type (#325)

* Add overwrite option to write builders (#318)

* Fix out of order Pig partition fields (#326)

* Add mapping to Iceberg for external name-based schemas (#338)

* Site: Fix broken link to Iceberg API (#333)

* Add forTable method for Avro WriteBuilder (#322)

* Remove multiple literal strings check rule for scala (#335)

* Fix invalid javadoc url in README.md (#336)

* Use UnicodeUtil.truncateString for Truncate transform. (#340)

This truncates by unicode codepoint instead of Java chars.

* Refactor metrics tests for reuse (#331)

* Spark: Add support for write-audit-publish workflows (#342)

* Avoid write failures if metrics mode is invalid (#301)

* Fix truncateStringMax in UnicodeUtil (#334)

Fixes #328, fixes #329.

Index to codePointAt should be the offset calculated by code points

* [Vectorization] Added batch sizing, switched to BufferAllocator, other minor style fixes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants