MINOR: Update to Gradle 6.5 and tweak build jvm config#8751
Merged
ijuma merged 4 commits intoapache:trunkfrom Jun 3, 2020
Merged
MINOR: Update to Gradle 6.5 and tweak build jvm config#8751ijuma merged 4 commits intoapache:trunkfrom
ijuma merged 4 commits intoapache:trunkfrom
Conversation
Member
Author
|
@omkreddy I'll rerun the numbers with 6.5 final tomorrow morning, but this should be ready for review otherwise. |
chia7712
approved these changes
Jun 3, 2020
Member
Author
|
@omkreddy Updated the PR description and made a few more changes. |
Member
Author
|
JDK 8/Scala 2.12 had an out of memory issue. |
Member
Author
|
retest this please |
Member
Author
|
Build passed. |
Member
Author
|
Cherry picked this to 2.6 as well since it's low risk and fixes a compilation speed regression. |
ijuma
added a commit
that referenced
this pull request
Jun 3, 2020
Gradle 6.5 includes a fix for gradle/gradle#12866, which affects the performance of Scala compilation. I profiled the scalac build with async profiler and 54% of the time was on GC even after the Gradle upgrade (it was more than 60% before), so I switched to the throughput GC (GC latency is less important for batch builds) and it was reduced to 38%. I also centralized the jvm configuration in `build.gradle` and simplified it a bit by removing the minHeapSize configuration from the test tasks. On my desktop, the time to execute clean builds with no cached Gradle daemon was reduced from 127 seconds to 97 seconds. With a cached daemon, it was reduced from 120 seconds to 88 seconds. The performance regression when we upgraded to Gradle 6.x was 27 seconds with a cached daemon (#7677 (comment)), so it should be fixed now. Gradle 6.4 with no cached daemon: ``` BUILD SUCCESSFUL in 2m 7s 115 actionable tasks: 112 executed, 3 up-to-date ./gradlew clean compileScala compileJava compileTestScala compileTestJava 1.15s user 0.12s system 0% cpu 2:08.06 total ``` Gradle 6.4 with cached daemon: ``` BUILD SUCCESSFUL in 2m 0s 115 actionable tasks: 111 executed, 4 up-to-date ./gradlew clean compileScala compileJava compileTestScala compileTestJava 0.95s user 0.10s system 0% cpu 2:01.42 total ``` Gradle 6.5 with no cached daemon: ``` BUILD SUCCESSFUL in 1m 46s 115 actionable tasks: 111 executed, 4 up-to-date ./gradlew clean compileScala compileJava compileTestScala compileTestJava 1.27s user 0.12s system 1% cpu 1:47.71 total ``` Gradle 6.5 with cached daemon: ``` BUILD SUCCESSFUL in 1m 37s 115 actionable tasks: 111 executed, 4 up-to-date ./gradlew clean compileScala compileJava compileTestScala compileTestJava 1.02s user 0.10s system 1% cpu 1:38.31 total ``` This PR with no cached Gradle daemon: ``` BUILD SUCCESSFUL in 1m 37s 115 actionable tasks: 81 executed, 34 up-to-date ./gradlew clean compileScala compileJava compileTestScala compileTestJava 1.27s user 0.10s system 1% cpu 1:38.70 total ``` This PR with cached Gradle daemon: ``` BUILD SUCCESSFUL in 1m 28s 115 actionable tasks: 111 executed, 4 up-to-date ./gradlew clean compileScala compileJava compileTestScala compileTestJava 1.02s user 0.10s system 1% cpu 1:29.35 total ``` Reviewers: Manikumar Reddy <manikumar.reddy@gmail.com>, Chia-Ping Tsai <chia7712@gmail.com>
ijuma
added a commit
that referenced
this pull request
Jun 4, 2020
…use_all_dns_ips-as-default * apache-github/trunk: KAFKA-9788; Use distinct names for transaction and group load time sensors (#8784) KAFKA-9514; The protocol generator generated useless condition when a field is made nullable and flexible version is used (#8793) MINOR: Update to Gradle 6.5 and tweak build jvm config (#8751) MINOR: Upgrade spotbugs and spotbugsPlugin (#8790) KAFKA-10089 The stale ssl engine factory is not closed after reconfigure (#8792) KAFKA-10080; Fix race condition on txn completion which can cause duplicate appends (#8782) KAFKA-10084: Fix EosTestDriver end offset (#8785) KAFKA-10083: fix failed testReassignmentWithRandomSubscriptionsAndChanges tests (#8786)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Gradle 6.5 includes a fix for gradle/gradle#12866, which
affects the performance of Scala compilation.
I profiled the scalac build with async profiler and 54% of the time was on GC
even after the Gradle upgrade (it was more than 60% before), so I switched to
the throughput GC (GC latency is less important for batch builds) and it
was reduced to 38%.
I also centralized the jvm configuration in
build.gradleand simplified it a bitby removing the minHeapSize configuration from the test tasks.
On my desktop, the time to execute clean builds with no cached Gradle daemon
was reduced from 127 seconds to 97 seconds. With a cached daemon, it was
reduced from 120 seconds to 88 seconds. The performance regression when
we upgraded to Gradle 6.x was 27 seconds with a cached daemon
(#7677 (comment)), so it
should be fixed now.
Gradle 6.4 with no cached daemon:
Gradle 6.4 with cached daemon:
Gradle 6.5 with no cached daemon:
Gradle 6.5 with cached daemon:
This PR with no cached Gradle daemon:
This PR with cached Gradle daemon:
Committer Checklist (excluded from commit message)