Skip to content

Add a new metric query/segments/count that is not emitted by default #11394

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jihoonson merged 6 commits into from
Jul 23, 2021
Merged

Add a new metric query/segments/count that is not emitted by default #11394

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
49c51cb
Add a new metric query/segments/count that is not emitted by default
Jun 29, 2021
ce43dcd
docs
Jul 6, 2021
08032ab
test the default implementation of the metric
Jul 6, 2021
f1b7e71
fix spelling error in docs
Jul 7, 2021
90e1e8b
document the fact that query retries will result in additional metric…
Jul 8, 2021
ca242ad
update using recommended text from @jihoonson
Jul 21, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion ...marks/src/test/java/org/apache/druid/benchmark/query/CachingClusteredClientBenchmark.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,7 @@
import org.apache.druid.segment.join.MapJoinableFactory;
import org.apache.druid.server.QueryStackTests;
import org.apache.druid.server.coordination.ServerType;
import org.apache.druid.server.metrics.NoopServiceEmitter;
import org.apache.druid.timeline.DataSegment;
import org.apache.druid.timeline.DataSegment.PruneSpecsHolder;
import org.apache.druid.timeline.SegmentId;
Expand Down Expand Up @@ -342,7 +343,8 @@ public <T, QueryType extends Query<T>> QueryToolChest<T, QueryType> getToolChest
processingConfig,
forkJoinPool,
QueryStackTests.DEFAULT_NOOP_SCHEDULER,
new MapJoinableFactory(ImmutableSet.of(), ImmutableMap.of())
new MapJoinableFactory(ImmutableSet.of(), ImmutableMap.of()),
new NoopServiceEmitter()
);
}

Expand Down
1 change: 1 addition & 0 deletions docs/operations/metrics.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,7 @@ Available Metrics
|`query/failed/count`|number of failed queries|This metric is only available if the QueryCountStatsMonitor module is included.||
|`query/interrupted/count`|number of queries interrupted due to cancellation.|This metric is only available if the QueryCountStatsMonitor module is included.||
|`query/timeout/count`|number of timed out queries.|This metric is only available if the QueryCountStatsMonitor module is included.||
|`query/segments/count`|This metric is not enabled by default. See the `QueryMetrics` Interface for reference regarding enabling this metric. Number of segments that will be touched by the query. In the broker, it makes a plan to distribute the query to realtime tasks and historicals based on a snapshot of segment distribution state. If there are some segments moved after this snapshot is created, certain historicals and realtime tasks can report those segments as missing to the broker. The broker will re-send the query to the new servers that serve those segments after move. In this case, those segments can be counted more than once in this metric.|Varies.|
|`sqlQuery/time`|Milliseconds taken to complete a SQL query.|id, nativeQueryIds, dataSource, remoteAddress, success.|< 1s|
|`sqlQuery/bytes`|number of bytes returned in SQL query response.|id, nativeQueryIds, dataSource, remoteAddress, success.| |

Expand Down
4 changes: 3 additions & 1 deletion ...rage-query/src/test/java/org/apache/druid/query/movingaverage/MovingAverageQueryTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,7 @@
import org.apache.druid.server.ClientQuerySegmentWalker;
import org.apache.druid.server.QueryStackTests;
import org.apache.druid.server.initialization.ServerConfig;
import org.apache.druid.server.metrics.NoopServiceEmitter;
import org.apache.druid.testing.InitializedNullHandlingTest;
import org.apache.druid.timeline.TimelineLookup;
import org.hamcrest.core.IsInstanceOf;
Expand Down Expand Up @@ -367,7 +368,8 @@ public String getFormatString()
},
ForkJoinPool.commonPool(),
QueryStackTests.DEFAULT_NOOP_SCHEDULER,
new MapJoinableFactory(ImmutableSet.of(), ImmutableMap.of())
new MapJoinableFactory(ImmutableSet.of(), ImmutableMap.of()),
new NoopServiceEmitter()
);

ClientQuerySegmentWalker walker = new ClientQuerySegmentWalker(
Expand Down
7 changes: 7 additions & 0 deletions processing/src/main/java/org/apache/druid/query/DefaultQueryMetrics.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -340,6 +340,13 @@ public QueryMetrics<QueryType> reportParallelMergeTotalCpuTime(long timeNs)
return this;
}

@Override
public QueryMetrics<QueryType> reportQueriedSegmentCount(long segmentCount)
{
// Don't emit by default.
return this;
}

@Override
public void emit(ServiceEmitter emitter)
{
Expand Down
5 changes: 5 additions & 0 deletions processing/src/main/java/org/apache/druid/query/QueryMetrics.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -270,6 +270,11 @@ public interface QueryMetrics<QueryType extends Query<?>>
*/
QueryMetrics<QueryType> reportQueryBytes(long byteCount);

/**
* Registeres "segments queried count" metric.
*/
QueryMetrics<QueryType> reportQueriedSegmentCount(long segmentCount);

/**
* Registers "wait time" metric.
*/
Expand Down
6 changes: 6 additions & 0 deletions processing/src/main/java/org/apache/druid/query/search/DefaultSearchQueryMetrics.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -291,6 +291,12 @@ public QueryMetrics reportParallelMergeTotalCpuTime(long timeNs)
return delegateQueryMetrics.reportParallelMergeTotalCpuTime(timeNs);
}

@Override
public QueryMetrics reportQueriedSegmentCount(long segmentCount)
{
return delegateQueryMetrics.reportQueriedSegmentCount(segmentCount);
}

@Override
public void emit(ServiceEmitter emitter)
{
Expand Down
7 changes: 7 additions & 0 deletions processing/src/test/java/org/apache/druid/query/DefaultQueryMetricsTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -148,5 +148,12 @@ public static void testQueryMetricsDefaultMetricNamesAndUnits(
actualEvent = cachingEmitter.getLastEmittedEvent().toMap();
Assert.assertEquals("query/node/bytes", actualEvent.get("metric"));
Assert.assertEquals(10L, actualEvent.get("value"));

// Here we are testing that Queried Segment Count does not get emitted by the DefaultQueryMetrics and the last
// metric remains as query/node/bytes
queryMetrics.reportQueriedSegmentCount(25).emit(serviceEmitter);
actualEvent = cachingEmitter.getLastEmittedEvent().toMap();
Assert.assertEquals("query/node/bytes", actualEvent.get("metric"));
Assert.assertEquals(10L, actualEvent.get("value"));
}
}
8 changes: 7 additions & 1 deletion server/src/main/java/org/apache/druid/client/CachingClusteredClient.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@
import org.apache.druid.java.util.common.guava.Sequence;
import org.apache.druid.java.util.common.guava.Sequences;
import org.apache.druid.java.util.emitter.EmittingLogger;
import org.apache.druid.java.util.emitter.service.ServiceEmitter;
import org.apache.druid.query.BySegmentResultValueClass;
import org.apache.druid.query.CacheStrategy;
import org.apache.druid.query.DruidProcessingConfig;
Expand Down Expand Up @@ -129,6 +130,7 @@ public class CachingClusteredClient implements QuerySegmentWalker
private final ForkJoinPool pool;
private final QueryScheduler scheduler;
private final JoinableFactoryWrapper joinableFactoryWrapper;
private final ServiceEmitter emitter;

@Inject
public CachingClusteredClient(
Expand All @@ -142,7 +144,8 @@ public CachingClusteredClient(
DruidProcessingConfig processingConfig,
@Merging ForkJoinPool pool,
QueryScheduler scheduler,
JoinableFactory joinableFactory
JoinableFactory joinableFactory,
ServiceEmitter emitter
)
{
this.warehouse = warehouse;
Expand All @@ -156,6 +159,7 @@ public CachingClusteredClient(
this.pool = pool;
this.scheduler = scheduler;
this.joinableFactoryWrapper = new JoinableFactoryWrapper(joinableFactory);
this.emitter = emitter;

if (cacheConfig.isQueryCacheable(Query.GROUP_BY) && (cacheConfig.isUseCache() || cacheConfig.isPopulateCache())) {
log.warn(
Expand Down Expand Up @@ -369,6 +373,8 @@ ClusterQueryResult<T> run(

query = scheduler.prioritizeAndLaneQuery(queryPlus, segmentServers);
queryPlus = queryPlus.withQuery(query);
queryPlus = queryPlus.withQueryMetrics(toolChest);
queryPlus.getQueryMetrics().reportQueriedSegmentCount(segmentServers.size()).emit(emitter);
Copy link
Copy Markdown
Contributor

@jihoonson jihoonson Jul 8, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RetryQueryRunner is responsible for re-routing the query to new homes of segments if they are moved during query processing. It uses CachingClusteredClient for re-routing. As a result, this can report the same segments multiple times if those segments are moved. I think this is OK and worth documenting.

Copy link
Copy Markdown
Contributor Author

@capistrant capistrant Jul 8, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jihoonson That retry code is unfamiliar to me at this time, Number of segments that will be touched by the query. If the query has to be retried, the metric will be reported for all retries as well as the original query. Is that a sane description of the metric in the metrics.md file?


final SortedMap<DruidServer, List<SegmentDescriptor>> segmentsByServer = groupSegmentsByServer(segmentServers);
LazySequence<T> mergedResultSequence = new LazySequence<>(() -> {
Expand Down
4 changes: 3 additions & 1 deletion server/src/test/java/org/apache/druid/client/CachingClusteredClientFunctionalityTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@
import org.apache.druid.segment.join.MapJoinableFactory;
import org.apache.druid.server.QueryStackTests;
import org.apache.druid.server.coordination.ServerType;
import org.apache.druid.server.metrics.NoopServiceEmitter;
import org.apache.druid.timeline.DataSegment;
import org.apache.druid.timeline.TimelineLookup;
import org.apache.druid.timeline.VersionedIntervalTimeline;
Expand Down Expand Up @@ -335,7 +336,8 @@ public int getMergePoolParallelism()
},
ForkJoinPool.commonPool(),
QueryStackTests.DEFAULT_NOOP_SCHEDULER,
new MapJoinableFactory(ImmutableSet.of(), ImmutableMap.of())
new MapJoinableFactory(ImmutableSet.of(), ImmutableMap.of()),
new NoopServiceEmitter()
);
}

Expand Down
4 changes: 3 additions & 1 deletion server/src/test/java/org/apache/druid/client/CachingClusteredClientPerfTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@
import org.apache.druid.server.QueryScheduler;
import org.apache.druid.server.coordination.ServerManagerTest;
import org.apache.druid.server.coordination.ServerType;
import org.apache.druid.server.metrics.NoopServiceEmitter;
import org.apache.druid.timeline.DataSegment;
import org.apache.druid.timeline.VersionedIntervalTimeline;
import org.apache.druid.timeline.partition.LinearShardSpec;
Expand Down Expand Up @@ -138,7 +139,8 @@ public void testGetQueryRunnerForSegments_singleIntervalLargeSegments()
Mockito.mock(DruidProcessingConfig.class),
ForkJoinPool.commonPool(),
queryScheduler,
NoopJoinableFactory.INSTANCE
NoopJoinableFactory.INSTANCE,
new NoopServiceEmitter()
);

Query<SegmentDescriptor> fakeQuery = makeFakeQuery(interval);
Expand Down
4 changes: 3 additions & 1 deletion server/src/test/java/org/apache/druid/client/CachingClusteredClientTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -125,6 +125,7 @@
import org.apache.druid.server.ServerTestHelper;
import org.apache.druid.server.coordination.ServerType;
import org.apache.druid.server.initialization.ServerConfig;
import org.apache.druid.server.metrics.NoopServiceEmitter;
import org.apache.druid.server.scheduling.ManualQueryPrioritizationStrategy;
import org.apache.druid.server.scheduling.NoQueryLaningStrategy;
import org.apache.druid.timeline.DataSegment;
Expand Down Expand Up @@ -2850,7 +2851,8 @@ public int getMergePoolParallelism()
NoQueryLaningStrategy.INSTANCE,
new ServerConfig()
),
new MapJoinableFactory(ImmutableSet.of(), ImmutableMap.of())
new MapJoinableFactory(ImmutableSet.of(), ImmutableMap.of()),
new NoopServiceEmitter()
);
}

Expand Down
4 changes: 3 additions & 1 deletion server/src/test/java/org/apache/druid/query/QueryRunnerBasedOnClusteredClientTestBase.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@
import org.apache.druid.segment.generator.SegmentGenerator;
import org.apache.druid.segment.join.MapJoinableFactory;
import org.apache.druid.server.QueryStackTests;
import org.apache.druid.server.metrics.NoopServiceEmitter;
import org.apache.druid.timeline.DataSegment;
import org.apache.druid.timeline.partition.NumberedShardSpec;
import org.joda.time.Interval;
Expand Down Expand Up @@ -145,7 +146,8 @@ public void setupTestBase()
),
ForkJoinPool.commonPool(),
QueryStackTests.DEFAULT_NOOP_SCHEDULER,
new MapJoinableFactory(ImmutableSet.of(), ImmutableMap.of())
new MapJoinableFactory(ImmutableSet.of(), ImmutableMap.of()),
new NoopServiceEmitter()
);
servers = new ArrayList<>();
}
Expand Down