Fixes in org.apache.kafka.connect.health#7482
Closed
941design wants to merge 121 commits intoapache:trunkfrom
941design:master
Closed
Fixes in org.apache.kafka.connect.health#7482941design wants to merge 121 commits intoapache:trunkfrom 941design:master
941design wants to merge 121 commits intoapache:trunkfrom
941design:master
Conversation
941design
commented
Oct 10, 2019
941design
commented
- Fixed Parameter order
- Fixed NPE check conditionals
In kafka-server-start.sh, use kafka.Kafka rather than io.confluent.support.metrics.SupportedKafka, for now.
The :core:compileScoverageScala gradle task keeps crashing with stack traces like these: > * What went wrong: > Execution failed for task ':core:compileScoverageScala'. > > java.lang.NullPointerException (no error message) > > * Try: > Run with --info or --debug option to get more log output. Run with --scan to get full insights. > > * Exception is: > org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':core:compileScoverageScala'. > at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:110) > at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:77) > at org.gradle.api.internal.tasks.execution.OutputDirectoryCreatingTaskExecuter.execute(OutputDirectoryCreatingTaskExecuter.java:51) > at org.gradle.api.internal.tasks.execution.SkipUpToDateTaskExecuter.execute(SkipUpToDateTaskExecuter.java:59) > at org.gradle.api.internal.tasks.execution.ResolveTaskOutputCachingStateExecuter.execute(ResolveTaskOutputCachingStateExecuter.java:54) > at org.gradle.api.internal.tasks.execution.ValidatingTaskExecuter.execute(ValidatingTaskExecuter.java:59) > at org.gradle.api.internal.tasks.execution.SkipEmptySourceFilesTaskExecuter.execute(SkipEmptySourceFilesTaskExecuter.java:101) > at org.gradle.api.internal.tasks.execution.FinalizeInputFilePropertiesTaskExecuter.execute(FinalizeInputFilePropertiesTaskExecuter.java:44) > at org.gradle.api.internal.tasks.execution.CleanupStaleOutputsExecuter.execute(CleanupStaleOutputsExecuter.java:91) > ... > Caused by: java.lang.NullPointerException > at scala.tools.nsc.backend.icode.GenICode$ICodePhase$Context.exitScope(GenICode.scala:1921) > at scala.tools.nsc.backend.icode.GenICode$ICodePhase.scala$tools$nsc$backend$icode$GenICode$ICodePhase$$genLoad(GenICode.scala:924) > at scala.tools.nsc.backend.icode.GenICode$ICodePhase.genLoadIf(GenICode.scala:373) > at scala.tools.nsc.backend.icode.GenICode$ICodePhase.scala$tools$nsc$backend$icode$GenICode$ICodePhase$$genLoad(GenICode.scala:536) > at scala.tools.nsc.backend.icode.GenICode$ICodePhase.scala$tools$nsc$backend$icode$GenICode$ICodePhase$$genStat(GenICode.scala:181) > at scala.tools.nsc.backend.icode.GenICode$ICodePhase$$anonfun$genStat$1.apply(GenICode.scala:155) > at scala.tools.nsc.backend.icode.GenICode$ICodePhase$$anonfun$genStat$1.apply(GenICode.scala:155) > at scala.collection.LinearSeqOptimized$class.foldLeft(LinearSeqOptimized.scala:124) > at scala.collection.immutable.List.foldLeft(List.scala:84) > at scala.tools.nsc.backend.icode.GenICode$ICodePhase.scala$tools$nsc$backend$icode$GenICode$ICodePhase$$genLoad(GenICode.scala:922) > at scala.tools.nsc.backend.icode.GenICode$ICodePhase.scala$tools$nsc$backend$icode$GenICode$ICodePhase$$genLoad(GenICode.scala:923) > at scala.tools.nsc.backend.icode.GenICode$ICodePhase$$anonfun$genLoadTry$1.apply(GenICode.scala:419) > at scala.tools.nsc.backend.icode.GenICode$ICodePhase$$anonfun$genLoadTry$1.apply(GenICode.scala:417) > at scala.tools.nsc.backend.icode.GenICode$ICodePhase$Context.Try(GenICode.scala:2095) > ... Disable this plugin temporarily so that "gradlew check" can succeed.
* apache-github/trunk: (23 commits) MINOR: SuppressScenarioTest should set StreamsConfig.STATE_DIR_CONFIG (#5826) KAFKA-7534: Error in flush calling close may prevent underlying store from closing (#5833) KAFKA-7352; Allow SASL Connections to Periodically Re-Authenticate (KIP-368) (#5582) MINOR: Remove unused commitSync in ConsoleConsumer (#5845) KAFKA-7532: Clean-up controller log when shutting down brokers (#5831) MINOR: Fix checkstyle error caused by #5781 MINOR: Simplify handling of KafkaProducer serializer overrides MINOR: add upgrade note for KIP-336 KAFKA-7535; KafkaConsumer doesn't report records-lag if isolation.level is read_committed KAFKA-5462: Add configuration to build custom SSL principal name (KIP-371) MINOR: Fix flaky assertion in ControllerIntegrationTest (#5829) MINOR: Fix undefined variable in Connect test MINOR: buffer should ignore caching (#5819) MINOR: fix docs typo (#5827) MINOR: Prohibit setting StreamsConfig commit.interval.ms to a negative value (#5809) KAFKA-7519 Clear pending transaction state when expiration fails (#5820) KAFKA-7501: Fix producer batch double deallocation when receiving message too large error on expired batch (#5807) KAFKA-7131: Update release script to generate announcement email text MINOR: Fix some typos MINOR: Less restrictive assertion in flaky BufferPool test (#5799) ...
* apache-github/trunk: MINOR: Remove unintentional tilde character from kafka-run-class.bat script MINOR: Remove duplicate `subscribe` call in ConsumerPerformance (#5828) MINOR: Bump timeout for sending records (#5843) MINOR: SuppressionIntegrationTest should set StreamsConfig.STATE_DIR_CONFIG (#5847) MINOR: Make log cleaner tests more efficient and less flaky (#5836)
* apache-github/trunk: KAFKA-7524: Recommend Scala 2.12 and use it for development (#5530)
* apache-github/trunk: (32 commits) KAFKA-7402: Implement KIP-376 AutoCloseable additions * MINOR: Catching null pointer exception for empty leader URL when assignment is null (#4798) KAFKA-6774; Improve the default group id behavior in KafkaConsumer (KIP-289) (#5877) KAFKA-7576; Fix shutdown of replica fetcher threads (#5875) HOTFIX: remove deprecated calls MINOR: Remove redundant SuppressIntegrationTests (#5896) MINOR: improve Puncutation JavaDocs and add runtime argument check (#5895) KAFKA-7584: StreamsConfig throws ClassCastException if max.in.flight.request.per.connect is specified as String (#5874) MINOR: Code cleanup in StreamsResetter (#5891) MINOR: Bump documentation version to 2.1 MINOR: Remove deprecated callers (#5911) MINOR: Avoid logging connector configuration in Connect framework (#5868) MINOR: Remove unused abstract function in test class (#5888) KAFKA-7514: Add threads to ConsumeBenchWorker (#5864) Trogdor: Fix /coordinator/tasks parameters to accept long values (#5905) KAFKA-7605; Retry async commit failures in integration test cases to fix flaky tests (#5890) KAFKA-7612: Fix javac warnings and enable warnings as errors (#5900) MINOR: Remove unused IteratorTemplate (#5903) KAFKA-7518: Fix FutureRecordMetadata.get when TimeUnit is not ms (#5815) KAFKA-7557: optimize LogManager.truncateFullyAndStartAt() (#5848) ...
* ak/trunk: MINOR: improve QueryableStateIntegrationTest (#5987) KAFKA-7235: Detect outdated control requests and bounced brokers using broker generation (#5821) KAFKA-4544: Add system tests for delegation token based authentication MINOR: hygene cleanup in TransactionManagerTest (#5951) MINOR: improve state directory test (#5961) KAFKA-7617: Add authorization primitives to security page KAFKA-7551: Refactor to create producer & consumer in the worker MINOR: Add logging to Connect SMTs MINOR: Fix handling of dummy record in EndToEndLatency tool KAFKA-7259; Remove deprecated ZKUtils usage from ZkSecurityMigrator KAFKA-7660: Fix child sensor memory leak (#5974) MINOR: Bump Gradle version to 5.0 (#5964) KAFKA-7660: fix parentSensors memory leak (#5953) MINOR: Fix ProducerPerformance bug when numRecords > Integer.MAX (#5956) Forward topic from console consumer to deserializer (#5704) KAFKA-7671: Stream-Global Table join should not reset repartition flag (#5959) MINOR: Support long maxMessages in Trogdor consume/produce bench workers (#5957) KAFKA-7037: Improve the topic command description of `--topic` option MINOR: Improve docs by adding ToC links to Monitoring KAFKA-7620: Fix restart logic for TTLs in WorkerConfigTransformer KAFKA-7389: Enable spotBugs with Java 11 and disable false positive warnings (#5943) KAFKA-7367: Streams should not create state store directories unless they are needed (#5696) MINOR: Add system test for optimization upgrades (#5912) KAFKA-7223: Suppression Buffer Metrics (#5795) KAFKA-7597: Add transaction support to ProduceBenchWorker (#5885) Trogdor: Add Task State filter to /coordinator/tasks endpoint (#5907) MINOR: Refactor code for restoring tasks (#5768) MINOR: Various javadoc improvement in clients and connect (#5878) MINOR: fix bootstrap-server typo in ReassignPartitionsCommand (#5941) KAFKA-7418: Add the missing '--help' option to Kafka commands (KIP-374) MINOR: increase system test kafka start timeout (#5934) MINOR: Update docs with out-dated context.schedule(...) examples (#5924) KAFKA-7528: Standardize on Min/Avg/Max Kafka metrics' default value - NaN (#5908) KAFKA-7536: Initialize TopologyTestDriver with non-null topic (#5923) KAFKA-7616; Make MockConsumer only add entries to the partition map returned by poll() if there are any records to return MINOR: Improve maven artifactory url in release.py (#5931) KAFKA-6567: Remove KStreamWindowReducer (#5922) MINOR: fix failing Streams system tests (#5928)
* apache-github/trunk: KAFKA-7704: MaxLag.Replica metric is reported incorrectly (#5998) KAFKA-7673: Upgrade rocksdb to 5.15.10 (#5985) KAFKA-6388; Recover from rolling an empty segment that already exists (#5986) MINOR: Safe string conversion to avoid NPEs KAFKA-7678: Avoid NPE when closing the RecordCollector (#5993) KAFKA-7420: Global store surrounded by read only implementation (#5865) KAFKA-7697: Process DelayedFetch without holding leaderIsrUpdateLock (#5999) MINOR: Improve GlobalKTable docs (#5996) KAFKA-7687; Print batch level information in DumpLogSegments when deep iterating (#5976) KAFKA-7702: Fix matching of prefixed ACLs to match single char prefix (#5994) KAFKA-7446: Fix the duration and instant validation messages. (#5930) MINOR: Update command line options in Authorization and ACLs documentation chapter (#5995) MINOR: Adding system test for named repartition topics (#5913)
* ak/trunk: KAFKA-7709: Fix ConcurrentModificationException when retrieving expired inflight batches on multiple partitions. (#6005)
* apache-github/trunk: MINOR: improve Streams checkstyle and code cleanup (#5954) KAFKA-6970: All standard state stores guarded with read only wrapper (#6016) KAFKA-7443: OffsetOutOfRangeException in restoring state store from changelog topic when start offset of local checkpoint is smaller than that of changelog topic (#5946) KAFKA-7610; Proactively timeout new group members if rebalance is delayed (#5962) KAFKA-7549; Old ProduceRequest with zstd compression does not return error to client (#5925) KAFKA-6036: Follow-up; cleanup sendOldValues logic ForwardingCacheFlushListener (#6017) KAFKA-6036: Local Materialization for Source KTable (#5779) MINOR: fix checkpoint write failure warning log (#6008) MINOR: Catch NoRecordsException in testCommaSeparatedRegex() test (#5944) MINOR: Specify character encoding in NetworkTestUtils (#5965) MINOR: Fix failing ConsumeBenchTest:test_multiple_consumers_specified_group_partitions_should_raise (#6015)
…sh suffix (#80) Since we want to retain the same base name for scripts as AK, it is less error-prone to just append `.sh` to the script name rather than get the directory and then add the script name. Also fixes issue with directories containing space. Reviewers: Ismael Juma <ismael@juma.me.uk>
* apache-upstream/trunk: MINOR: Update README.md with Gradle 5+ requirement (#6039) MINOR: Include additional detail in fetch error message (#6036) MINOR: improve Streams error message (#5975) KAFKA-2334; Guard against non-monotonic offsets in the client (#5991) MINOR: Replace tbd with the actual link for out-of-ordering data (#6035) MINOR: Update documentation for internal changelog when using table(). (#6021) MINOR:Start processor inside verify message (#6029) Fix the missing ApiUtils tests in streams module. (#6003) KAFKA-7655 Metadata spamming requests from Kafka Streams under some circumstances, potential DOS (#5929) KAFKA-7223: document suppression buffer metrics (#6024) KAFKA-7712; Remove channel from Selector before propagating exception (#6023) KAFKA-7705; Fix and simplify producer config in javadoc example (#6000) MINOR: Add unit for max latency in ProducerPerformance output (#6014)
* apache/trunk: MINOR: Add test demonstrating re-use of KGroupedStream with Optimizations enabled (#6050) MINOR: Bump admin client retries for creating repartition topics (#6063) MINOR: Increase throughput for VerifiableProducer in test (#6060) KAFKA-7054; Kafka describe command should throw topic doesn't exist exception MINOR: Hygiene fixes in KafkaFutureImpl (#5098) MINOR: Switch anonymous classes to lambda expressions in tools module KAFKA-7759; Disable WADL output in the Connect REST API (#6051) KAFKA-7762; Update KafkaConsumer Javadoc examples to use poll(Duration timeout) API KAFKA-7742; Fixed removing hmac entry for a token being removed from DelegationTokenCache MINOR: Stabilization fixes broker down test trunk (#6043)
* apache-kafka/trunk: MINOR: Fix doc format in upgrade notes MINOR: Improve exception messages in FileChannelRecordBatch (#6068) KAFKA-3832; Kafka Connect's JSON Converter never outputs a null value (#6027)
* ak/trunk: (41 commits) KAFKA-7808: AdminClient#describeTopics should not throw InvalidTopic if topic name is not found (#6124) HOTFIX: Compilation error in CommandLineUtils (#6131) KAFKA-7790: Fix Bugs in Trogdor Task Expiration (#6103) MINOR: Need to have same wait as verify timeout broker upgrade downgrade (#6127) KAFKA-6627: Prevent config default values overriding ones specified through --producer-property on command line. (#6084) KAFKA-7741: Streams exclude javax dependency (#6121) MINOR: Remove throwing exception if not found from describe topics (#6112) KAFKA-5994; Log ClusterAuthorizationException for all ClusterAction requests MINOR: Update dependencies for Kafka 2.2 (#6116) MINOR: Add 2.1 version metadata upgrade (#6111) KAFKA-7786; Ignore OffsetsForLeaderEpoch response if epoch changed while request in flight (#6101) MINOR: Log successful/failed authentications with socket information (#5856) MINOR: code cleanup (#6053) MINOR: code cleanup (#6057) MINOR: code cleanup (#6055) KAFKA-7799; Fix flaky test RestServerTest.testCORSEnabled (#6106) KAFKA-6833; Producer should await metadata for unknown partitions (#6073) MINOR: Put states in proper order, increase timeout for starting (#6105) MINOR: code cleanup (#6056) KAFKA-7253; The returned connector type is always null when creating connector (#5470) ...
Reviewers: Rajini Sivaram <rajinisivaram@googlemail.com>
… compatibility test (#7023) Connect tests were using String version for KafkaService instead of the expected KafkaVersion object. This broke due to recent changes to KafkaVersion. It turns out that the tests with String version were running compatibility tests against `dev` brokers rather than the older broker versions they were expecting to run against. When version was fixed, tests using 0.9.0.1 brokers started failing since new clients are not compatible with 0.9.0.1 brokers. So this PR fixes version parameter and removes the two tests against 0.9.0.1 brokers. Reviewers: Ismael Juma <ismael@juma.me.uk>, Rajini Sivaram <rajinisivaram@googlemail.com>
No conflicts, however the return type for `zkClient.getPartitionAssignmentForTopics` changes from a `scala.collection.immutable.Map` to `scala.collection.Map`. This change was accounted for in `KafkaUtilities.java`
This reverts the changes done in: confluentinc@e5a6ed3
This stopped working when we migrated this code to Gradle in the
`confluentinc/kafka` repository. Previously, the version.properties
file was populated by `Maven`. Rely on post-processing of the kafka
version instead.
Without this change, the basic collector is basically broken as the
server will reject messages that have `${project.version}` as the
version.
Reviewers: Manikumar Reddy <manikumar.reddy@gmail.com>
…re (#531) (#215) We wish to return a yellow build on a test failure, but wish to fail the build completely when the gradle executors do not finish cleanly. This change will achieve that. (cherry picked from commit 334acf48d40e6a3c627cb71990fa91e1616e4921) -- This also changes max number of forks to 4, to improve build stability
… coverage (#7159) LogConfig.getConfigValue would throw a NoSuchElementException if any log config was defined without a server default mapping. Added a unit test for `getConfigValue` and a sanity test for `toHtml`/`toRst`/`toEnrichedRst`, which were previously not exercised during the test suite. Reviewers: Jason Gustafson <jason@confluent.io>, José Armando García Sancio <jsancio@users.noreply.github.com>
The avro upgrade was needed to fix the following error during ':support-metrics-client:generateAvro': Caused by: org.apache.velocity.exception.MethodInvocationException: Variable $velocityCount has not been set at /org/apache/avro/compiler/specific/templates/java/classic/record.vm[line 82, column 150] Conflicts: * gradle.properties -> trivial fix, `scalaVersion` is next to `version`, which is different in ccs kafka * gradle/dependencies.gradle -> reduce divergence with apache kafka by using `httpclient` instead of `httpcomponents` from the `versions` array. Remove unused `httpmime` and `httpcomponents` from said array. * apache-github/trunk: MINOR: Update dependencies for Kafka 2.4 (#7126) KAFKA-8599: Use automatic RPC generation in ExpireDelegationToken MINOR: Upgrade jackson-databind to 2.9.9.3 (#7125) MINOR: some small style fixes to RoundRobinPartitioner KAFKA-8736: Streams performance improvement, use isEmpty() rather than size() == 0 (#7164) Minor: Refactor methods to add metrics to sensor in `StreamsMetricsImpl` (#7161)
* apache-github/trunk: MINOR: Ignore dynamic log4j log level tests (#7183) KAFKA-8748: Fix flaky testDescribeLogDirsRequest (#7182) KAFKA-8598: Use automatic RPC generation in RenewDelegationToken KAFKA-8179: Part 3, Add PartitionsLost API for resetGenerations and metadata/subscription change (#6884)
Performed the following change for the tests to passed: 1. Included scalatest for support-metrics-clients 2. Changed to some of the types in KafkaUtilities since the signature for KafkaZkClient changed upstream.
As per the new Slack changes we want all the notifications from bots which are not immediately actionable to go to a different channel. Moving the jenkins build alerts to #kafka-warn
Conflicts: * .gitignore: addition of clients/src/generated-test was near local additions for support-metrics. * checkstyle/suppressions.xml: upstream refactoring of exclusions for generator were near the local changes for support-metrics. * gradle.properties: scala version bump caused a minor conflict due to the kafka version change locally. gradle/dependencies.gradle: bcpkix version bump was near avro additions in the local version. * apache-github/trunk: (49 commits) KAFKA-8471: Replace control requests/responses with automated protocol (#7353) MINOR: Don't generate unnecessary strings for debug logging in FetchSessionHandler (#7394) MINOR:fixed typo and removed outdated varilable name (#7402) KAFKA-8934: Create version file during build for Streams (#7397) KAFKA-8319: Make KafkaStreamsTest a non-integration test class (#7382) KAFKA-6883: Add toUpperCase support to sasl.kerberos.principal.to.local rule (KIP-309) KAFKA-8907; Return topic configs in CreateTopics response (KIP-525) (#7380) MINOR: Address review comments for KIP-504 authorizer changes (#7379) MINOR: add versioning to request and response headers (#7372) KAFKA-7273: Extend Connect Converter to support headers (#6362) MINOR: improve the Kafka RPC code generator (#7340) MINOR: Improve the org.apache.kafka.common.protocol code (#7344) KAFKA-8880: Docs on upgrade-guide (#7385) KAFKA-8179: do not suspend standby tasks during rebalance (#7321) KAFKA-8580: Compute RocksDB metrics (#7263) KAFKA-8880: Add overloaded function of Consumer.committed (#7304) HOTFIX: fix Kafka Streams upgrade note for broker backward compatibility (#7363) KAFKA-8848; Update system tests to use new AclAuthorizer (#7374) MINOR: remove unnecessary null check (#7299) KAFKA-6958: Overload methods for group and windowed stream to allow to name operation name using the new Named class (#6413) ...
Contributor
|
The bugs have been fixed by: #7392 Closing here without merging. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.