Skip to content

Register lookup module with overlord and middle manager. #7082

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
scrawfor wants to merge 3 commits into from
Closed

Register lookup module with overlord and middle manager. #7082

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
011f7d4
Register lookup module with middle manager and overlord.
pincscrawfor Feb 14, 2019
0d2eba1
Add configuration examples.
pincscrawfor Feb 15, 2019
59f715f
Register Lookup Modules.
pincscrawfor Feb 15, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions examples/conf/druid/middleManager/runtime.properties
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,3 +36,7 @@ druid.indexer.fork.property.druid.processing.numThreads=2

# Hadoop indexing
druid.indexer.task.hadoopWorkingPath=var/druid/hadoop-tmp

# Disable lookups on middleManager, but enable on peon.
druid.lookup.enableLookupSyncOnStartup=false
druid.indexer.fork.property.druid.lookup.enableLookupSyncOnStartup=true
4 changes: 4 additions & 0 deletions examples/quickstart/tutorial/conf/druid/middleManager/runtime.properties
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,3 +36,7 @@ druid.indexer.fork.property.druid.processing.numThreads=2

# Hadoop indexing
druid.indexer.task.hadoopWorkingPath=var/druid/hadoop-tmp

# Disable lookups on middleManager, but enable on peon.
druid.lookup.enableLookupSyncOnStartup=false
druid.indexer.fork.property.druid.lookup.enableLookupSyncOnStartup=true
4 changes: 3 additions & 1 deletion services/src/main/java/org/apache/druid/cli/CliMiddleManager.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,7 @@
import org.apache.druid.indexing.worker.http.TaskManagementResource;
import org.apache.druid.indexing.worker.http.WorkerResource;
import org.apache.druid.java.util.common.logger.Logger;
import org.apache.druid.query.lookup.LookupModule;
import org.apache.druid.segment.realtime.firehose.ChatHandlerProvider;
import org.apache.druid.server.DruidNode;
import org.apache.druid.server.initialization.jetty.JettyServerInitializer;
Expand Down Expand Up @@ -163,7 +164,8 @@ public WorkerNodeService getWorkerNodeService(WorkerConfig workerConfig)
}
},
new IndexingServiceFirehoseModule(),
new IndexingServiceTaskLogsModule()
new IndexingServiceTaskLogsModule(),
new LookupModule()
Copy link
Copy Markdown
Member

@clintropolis clintropolis Mar 12, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#7222 introduces LookupSerdeModule which I think might be nicer behavior for middle manager and overlord than using LookupModule and configuring it not to load, but it will require this PR to wait until the other is merged.

Copy link
Copy Markdown
Contributor

@jon-wei jon-wei Apr 20, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@scrawfor #7222 was merged, can you please rebase when you get a chance?

);
}
}
4 changes: 3 additions & 1 deletion services/src/main/java/org/apache/druid/cli/CliOverlord.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,7 @@
import org.apache.druid.indexing.overlord.supervisor.SupervisorResource;
import org.apache.druid.indexing.worker.config.WorkerConfig;
import org.apache.druid.java.util.common.logger.Logger;
import org.apache.druid.query.lookup.LookupModule;
import org.apache.druid.segment.realtime.firehose.ChatHandlerProvider;
import org.apache.druid.server.audit.AuditManagerProvider;
import org.apache.druid.server.coordinator.CoordinatorOverlordServiceConfig;
Expand Down Expand Up @@ -331,7 +332,8 @@ private void configureOverlordHelpers(Binder binder)
}
},
new IndexingServiceFirehoseModule(),
new IndexingServiceTaskLogsModule()
new IndexingServiceTaskLogsModule(),
new LookupModule()
);
}

Expand Down