Adding example config files for indexer process. Also added status/selfDiscovered endpoint to indexer for self discovery#10679
Merged
xvrl merged 1 commit intoapache:masterfrom Dec 15, 2020
Conversation
xvrl
reviewed
Dec 14, 2020
harinirajendran
force-pushed
the
master
branch
from
c15cc87 to
4c7416c
Compare
xvrl
reviewed
Dec 14, 2020
Member
There was a problem hiding this comment.
the corresponding middleManager config has 4 tasks, should we match that for consistency?
Contributor
Author
There was a problem hiding this comment.
Sure! I can update it to match with MM config.
harinirajendran
changed the title
harinirajendran
changed the title
xvrl
reviewed
Dec 14, 2020
Member
There was a problem hiding this comment.
given we have 1 processing thread per task in the example middle-manager config, I would set this to equal the worker capacity to have a comparable config.
xvrl
reviewed
Dec 14, 2020
Comment on lines
2
to
4
Member
There was a problem hiding this comment.
we probably need more heap to match our middle-manager config, which had 1GB per task.
Contributor
Author
There was a problem hiding this comment.
Made it 4g and also increased druid.processing.buffer.sizeBytes to 500MiB . Also increased -Xms and -Xmx setting to be 4x.
harinirajendran
changed the title
harinirajendran
force-pushed
the
master
branch
from
4c7416c to
6c53d8b
Compare
harinirajendran
force-pushed
the
master
branch
from
6c53d8b to
9a7d32d
Compare
xvrl
reviewed
Dec 14, 2020
harinirajendran
force-pushed
the
master
branch
from
9a7d32d to
a37824a
Compare
…lfDiscovered endpoint to indexer for self discovery of indexer.
harinirajendran
force-pushed
the
master
branch
from
a37824a to
062572a
Compare
xvrl
approved these changes
Dec 14, 2020
Member
xvrl
left a comment
xvrl
left a comment
There was a problem hiding this comment.
LGTM, cc @himanshug in case he has anything to add.
Member
|
merging since test failures appear unrelated. |
harinirajendran
added a commit
to confluentinc/druid
that referenced
this pull request
Dec 15, 2020
…ndexer (apache#10679) Added the status/selfDiscovered endpoint to indexer. Per the api-reference doc, all services support status/selfDiscovered endpoint. So this change would fix that expected behavior. Also added example config files for indexer process that can be used to spin up the indexer process. (cherry picked from commit c2e26d2)
harinirajendran
added a commit
to confluentinc/druid
that referenced
this pull request
Dec 15, 2020
Add status/selfDiscovered endpoint to indexer for self discovery of indexer (apache#10679)
JulianJaffePinterest
pushed a commit
to JulianJaffePinterest/druid
that referenced
this pull request
Jan 22, 2021
…ndexer (apache#10679) Added the status/selfDiscovered endpoint to indexer. Per the api-reference doc, all services support status/selfDiscovered endpoint. So this change would fix that expected behavior. Also added example config files for indexer process that can be used to spin up the indexer process.
harinirajendran
added a commit
to confluentinc/druid
that referenced
this pull request
Apr 19, 2021
…ndexer (apache#10679) Added the status/selfDiscovered endpoint to indexer. Per the api-reference doc, all services support status/selfDiscovered endpoint. So this change would fix that expected behavior. Also added example config files for indexer process that can be used to spin up the indexer process. (cherry picked from commit c2e26d2)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
In this PR I have added example config files that can be used to spin up the indexer process. And have also added the
status/selfDiscoveredendpoint to indexer. Per the api-reference doc, all services supportstatus/selfDiscoveredendpoint. So this PR would fix that expected behavior.Code changes:
Since the SelfDiscovered endpoint is registered in the middle manager here, I followed the similar pattern in CliIndexer.java file as well
Testing
Built my own docker image, added the following in
docker-compose.ymlfileBrought all the services up
Made
status/selfDiscoveredcurl call to indexer running on port 8092 and got back 200 OK.