[Backport] Coordinator primary segment assignment fix#5949
Closed
jihoonson wants to merge 82 commits intoapache:masterfrom
Closed
[Backport] Coordinator primary segment assignment fix#5949jihoonson wants to merge 82 commits intoapache:masterfrom
jihoonson wants to merge 82 commits intoapache:masterfrom
Conversation
* Add better logging messages in lookups startup on query nodes * Make sure list is mutable * Move list to be with other `final` variables
* Fix broken KafkaEmitterConfig parsing This was a regression introduced in apache#4722 KafkaEmitterConfig property names have dot(.) in the name of properties and JsonConfigurator behavior was changed to not support that. Added a test and fixed parsing of properties that have dot(.) in property names * Fix test failure
…pache#5209) * support replaceExisting parameter for segments pushers * code review changes * code review changes
* Upgrade to Calcite 1.15.0 * Use Filtration.eternity()
…pache#5215) (apache#5232) * Throw away rows with timestamps beyond long bounds in kafka indexing * PR comments
* Encrypting MySQL connections * Update docs * Make verifyServerCertificate a configurable parameter * Change password parameter and doc update * Make server certificate verification disabled by default * Update tostring * Update docs * Add check for trust store passwords * Add warning for null password
[Backport] Fix state check bug in Kafka Index Task apache#5204
…ng DST tran… (apache#5252) * fix timewarp query results when using timezones and crossing DST transitions (apache#5157) * timewarp and timezones changes: * `TimewarpOperator` will now compensate for daylight savings time shifts between date translation ranges for queries using a `PeriodGranularity` with a timezone defined * introduces a new abstract query type `TimeBucketedQuery` for all queries which have a `Granularity` (100% not attached to this name). `GroupByQuery`, `SearchQuery`, `SelectQuery`, `TimeseriesQuery`, and `TopNQuery` all extend `TimeBucke tedQuery`, cutting down on some duplicate code and providing a mechanism for `TimewarpOperator` (and anything else) that needs to be aware of granularity * move precondition check to TimeBucketedQuery, add Granularities.nullToAll, add getTimezone to TimeBucketQuery * formatting * more formatting * unused import * changes: * add 'getGranularity' and 'getTimezone' to 'Query' interface * merge 'TimeBucketedQuery' into 'BaseQuery' * fixup tests from resulting serialization changes * dedupe * fix after merge * suppress warning * Fix compile issue
…druid-processing (apache#5275) (apache#5283) * Fix bugs in ImmutableRTree; optimize ImmmutableRTreeObjectStrategy.writeTo(); Merge bytebuffer-collections module into druid-processing * Remove unused declaration * Fix another bug
… druid router (apache#5274) (apache#5285) * Fix rewrite of queryPath for encoded joda intervals as query param on druid router * fix checkstyle * fix comment
…aler minNumWorkers value (apache#5324) * fix RemoteTaskRunner terminating lazy workers below autoscaler minNumWorkers value * add comment
…ache#5356) * Discard rows with unparseable numeric dimensions * PR comments * Don't throw away entire row on parse exception * PR comments * Fix import
…pache#5357) * Fix two improper casts in HavingSpecMetricComparator. Fixes two things: 1. An improper double-to-long cast when comparing double metrics to any kind of value, which was a regression from apache#4883. 2. An improper double-to-long cast when comparing a long/int metric to a double/float value: the value was cast to long/int, drawing strange conclusions like int 100 matching a havingSpec of equalTo(100.5). * Add comments. * Remove extraneous comment. * Simplify code a bit.
…rCacheManager (apache#5344) (apache#5360) * Fix races in LookupSnapshotTaker, CoordinatorPollingBasicAuthenticatorCacheManager. Both were susceptible to the following conditions: 1. Two JVMs on the same machine (perhaps two peons) could conflict by one reading while the other was writing, or by writing to the file at the same time. 2. One JVM could partially write a file, then crash, leaving a truncated file. * Use StringUtils.format
…) (apache#5361) Similar to apache#5344 but for the authorizer instead of the authenticator.
apache#5367) Prevents conflicts if two processes on the same machine use the same lookup snapshot directory but are in different tiers.
…druid java-util (apache#5289) (apache#5387) * Add metamx emitter, http clients, and metrics packages to druid java-util (apache#5289) * Add metamx java-util emitter, http clients, and metrics packages to druid java-util * Remove metamx java-util from pom.xml files * Checkstyle fixes * Import fix * TeamCity inspection fixes * Use slf4j, move some version defs to master pom.xml * Use parent jvm-attach-api and maven-surefire-plugin versions * Add ] to log msg, suppress inspection * Remove extra files, fix HttpPostEmitter compile
…#5393) * fix segment info in Kafka indexing service docs * review updates
…OutOfMemoryError) during onSealExclusive() (apache#5386) (apache#5395)
…penderatorDriver (apache#5397)
…5630) (apache#5633) Once a buffer is acquired, we need to make sure to release it if an exception is thrown before the closeable iterator is created.
* Optimize coordinator loadStatus * Add comment * Fix teamcity * Checkstyle * More checkstyle * Checkstyle
…eCounter (apache#5592) (apache#5642) * Replace EmittedBatchCounter and UpdateCounter with (both not safe for concurrent increments/updates) with ConcurrentAwaitableCounter (safe for concurrent increments) * Fixes * Fix EmitterTest * Added Javadoc and make awaitCount() to throw exceptions on wrong count instead of masking errors
…he#5654) * Fix HTTP OPTIONS request auth handling (apache#5638) * Fix HTTP OPTIONS request auth handling * PR comment * More PR comments * Fix * PR comment * Compile fixes * Compile
[Backport] Add missing doc for automatic pendingSegments cleanup
* support unique segment file paths * forbiddenapis * code review changes * code review changes * code review changes * checkstyle fix
* Adding decoration method to proxy servlet Change-Id: I872f9282fb60bfa20524271535980a36a87b9621 * moving the proxy request decoration to authenticators Change-Id: I7f94b9ff5ecf08e8abf7169b58bc410f33148448 * added docs Change-Id: I901543e52f0faf4666bfea6256a7c05593b1ae70 * use the authentication result to decorate request Change-Id: I052650de9cd02b4faefdbcdaf2332dd3b2966af5 * adding authenticated by name Change-Id: I074d2933460165feeddb19352eac9bd0f96f42ca * ensure that authenticator is not null Change-Id: Idb58e308f90db88224a06f3759114872165b24f5 * fix types and minor bug Change-Id: I6801d49a05d5d8324406fc0280286954eb66db10 * fix typo Change-Id: I390b12af74f44d760d0812a519125fbf0df4e97b * use actual type names Change-Id: I62c3ee763363781e52809ec912aafd50b8486b8e * set authenitcatedBy to null for AutheticationResults created by Escalator. Change-Id: I4a675c372f59ebd8a8d19c61b85a1e4bf227a8ba
…g on startup. (apache#5735) (apache#5755) The "lock" object was used to synchronize start/stop as well as synchronize removals from segmentsToDelete (when a segment is done dropping). This could cause a deadlock if a segment-load throws an exception during loadLocalCache. loadLocalCache is run by start() while it holds the lock, but then it spawns loading threads, and those threads will try to acquire the "segmentsToDelete" lock if they want to drop a corrupt segments. I don't see any reason for these two locks to be the same lock, so I split them.
* [Backport] Fix metrics for inserting segments * fix build
* Fix defaultQueryTimeout - set default timeout in query context before query fail time is evaluated Remove unused import * Address failing checks * Addressing code review comments * Removed line that was no longer used
…apache#5790) (apache#5942) This feature was introduced in apache#5717 but it didn't work in production because this magical rewriter code wasn't also modified. Now, it is.
…ce (apache#5943) * Allow reordered segment allocation in kafka indexing service (apache#5805) * Allow reordered segment allocation in kafka indexing service * address comments * fix a bug * commit remaining changes
* fix issue where assign primary assigns segments to all historical servers in cluster * fix test * add test to ensure primary assignment will not assign to another server while loading is in progress
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
9 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Backport of #5532 to 0.12.2.