Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 33 additions & 24 deletions processing/src/main/java/io/druid/query/QueryMetrics.java
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
import com.metamx.emitter.service.ServiceEmitter;
import io.druid.collections.bitmap.BitmapFactory;
import io.druid.query.filter.Filter;
import io.druid.query.search.SearchQueryMetricsFactory;
import org.joda.time.Interval;

import java.util.List;
Expand Down Expand Up @@ -110,30 +111,35 @@
*
* Making subinterfaces of QueryMetrics for emitting custom dimensions and/or metrics for specific query types
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of referencing TopN, GroupBy and Timeseries as something that should not be taken as examples of following this procedure, in the end of the description of this procedure, please reference Search and Select as examples of this procedure implemented. Also please add notes about empty or exception throwing of query() and other "pre-query-execution-time" methods to this procedure.

* -----------------------------------------------------------------------------------------------------------
* If a query type (e. g. {@link io.druid.query.search.SearchQuery} (it's runners) needs to emit custom
* dimensions and/or metrics which doesn't make sense for all other query types, the following steps should be executed:
* 1. Create `interface SearchQueryMetrics extends QueryMetrics` (here and below "Search" is the query type) with
* additional methods (see "Adding new methods" section above).
*
* 2. Create `class DefaultSearchQueryMetrics implements SearchQueryMetrics`. This class should implement extra methods
* from SearchQueryMetrics interfaces with empty bodies, AND DELEGATE ALL OTHER METHODS TO A QueryMetrics OBJECT,
* provided as a sole parameter in DefaultSearchQueryMetrics constructor.
*
* 3. Create `interface SearchQueryMetricsFactory` with a single method
* `SearchQueryMetrics makeMetrics(SearchQuery query);`.
*
* 4. Create `class DefaultSearchQueryMetricsFactory implements SearchQueryMetricsFactory`, which accepts {@link
* GenericQueryMetricsFactory} as injected constructor parameter, and implements makeMetrics() as
* `return new DefaultSearchQueryMetrics(genericQueryMetricsFactory.makeMetrics(query));`
*
* 5. Inject and use SearchQueryMetricsFactory instead of {@link GenericQueryMetricsFactory} in {@link
* io.druid.query.search.SearchQueryQueryToolChest}.
*
* 6. Establish injection of SearchQueryMetricsFactory using config and provider method in QueryToolChestModule
* (see how it is done in QueryToolChestModule for existing query types with custom metrics, e. g. {@link
* io.druid.query.topn.TopNQueryMetricsFactory}), if the query type belongs to the core druid-processing, e. g.
* SearchQuery. If the query type defined in an extension, you can specify
* `binder.bind(ScanQueryMetricsFactory.class).to(DefaultScanQueryMetricsFactory.class)` in the extension's
* If a query type (e. g. {@link io.druid.query.metadata.metadata.SegmentMetadataQuery} (it's runners) needs to emit
* custom dimensions and/or metrics which doesn't make sense for all other query types, the following steps should be
* executed:
*
* 1. Create `interface SegmentMetadataQueryMetrics extends QueryMetrics` (here and below "SegmentMetadata" is the
* query type) with additional methods (see "Adding new methods" section above).
*
* 2. Create `class DefaultSegmentMetadataQueryMetrics implements SegmentMetadataQueryMetrics`. This class should
* implement extra methods from SegmentMetadataQueryMetrics interfaces with empty bodies, AND DELEGATE ALL OTHER
* METHODS TO A QueryMetrics OBJECT, provided as a sole parameter in DefaultSegmentMetadataQueryMetrics constructor.
*
* NOTE: query(), dataSource(), queryType(), interval(), hasFilters(), duration() and queryId() methods or any
* "pre-query-execution-time" methods should either have a empty body or throw exception.
*
* 3. Create `interface SegmentMetadataQueryMetricsFactory` with a single method
* `SegmentMetadataQueryMetrics makeMetrics(SegmentMetadataQuery query);`.
*
* 4. Create `class DefaultSegmentMetadataQueryMetricsFactory implements SegmentMetadataQueryMetricsFactory`,
* which accepts {@link GenericQueryMetricsFactory} as injected constructor parameter, and implements makeMetrics() as
* `return new DefaultSegmentMetadataQueryMetrics(genericQueryMetricsFactory.makeMetrics(query));`
*
* 5. Inject and use SegmentMetadataQueryMetricsFactory instead of {@link GenericQueryMetricsFactory} in
* {@link io.druid.query.metadata.SegmentMetadataQueryQueryToolChest}.
*
* 6. Establish injection of SegmentMetadataQueryMetricsFactory using config and provider method in
* QueryToolChestModule (see how it is done in {@link io.druid.guice.QueryToolChestModule} for existing query types
* with custom metrics, e. g. {@link SearchQueryMetricsFactory}), if the query type
* belongs to the core druid-processing, e. g. SegmentMetadataQuery. If the query type defined in an extension, you
* can specify `binder.bind(ScanQueryMetricsFactory.class).to(DefaultScanQueryMetricsFactory.class)` in the extension's
* Guice module, if the query type is defined in an extension, e. g. ScanQuery. Or establish similar configuration,
* as for the core query types.
*
Expand All @@ -146,6 +152,9 @@
* dimensions than the default generic QueryMetrics. So those subinterfaces shouldn't be taken as direct examples for
* following the plan specified above.
*
* Refer {@link SearchQueryMetricsFactory}
* and {@link io.druid.query.select.SelectQueryMetricsFactory} as an implementation example of this procedure.
*
* @param <QueryType>
*/
public interface QueryMetrics<QueryType extends Query<?>>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ public void query(GroupByQuery query)
numDimensions(query);
numMetrics(query);
numComplexMetrics(query);
granularity(query);
}

@Override
Expand All @@ -58,4 +59,10 @@ public void numComplexMetrics(GroupByQuery query)
int numComplexAggs = DruidMetrics.findNumComplexAggs(query.getAggregatorSpecs());
setDimension("numComplexMetrics", String.valueOf(numComplexAggs));
}

@Override
public void granularity(GroupByQuery query)
{
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add comment "Don't emit by default". Same in other places

//Don't emit by default
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -42,4 +42,9 @@ public interface GroupByQueryMetrics extends QueryMetrics<GroupByQuery>
* method.
*/
void numComplexMetrics(GroupByQuery query);

/**
* Sets the granularity of {@link GroupByQuery#getGranularity()} of the given query as dimension.
*/
void granularity(GroupByQuery query);
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,240 @@
/*
* Licensed to Metamarkets Group Inc. (Metamarkets) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. Metamarkets licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/

package io.druid.query.search;

import com.metamx.emitter.service.ServiceEmitter;
import io.druid.collections.bitmap.BitmapFactory;
import io.druid.java.util.common.ISE;
import io.druid.query.BitmapResultFactory;
import io.druid.query.Query;
import io.druid.query.QueryMetrics;
import io.druid.query.filter.Filter;
import org.joda.time.Interval;

import java.util.List;

/**
* This class is implemented with delegation to another QueryMetrics for compatibility, see "Making subinterfaces of
* QueryMetrics for emitting custom dimensions and/or metrics for specific query types" section in {@link QueryMetrics}
* javadoc.
*/
public class DefaultSearchQueryMetrics implements SearchQueryMetrics
{
private QueryMetrics<Query<?>> delegateQueryMetrics;


// queryMetrics.query(query) must already be called on the provided queryMetrics.
public DefaultSearchQueryMetrics(QueryMetrics<Query<?>> queryMetrics)
{
this.delegateQueryMetrics = queryMetrics;
}

@Override
public void query(SearchQuery query)
{
//delegateQueryMetrics.query(query) must already be called on the provided queryMetrics.
}

@Override
public void dataSource(SearchQuery query)
{
throw new ISE("Unsupported method in default query metrics implementation.");
}

@Override
public void queryType(SearchQuery query)
{
throw new ISE("Unsupported method in default query metrics implementation.");
}

@Override
public void interval(SearchQuery query)
{
throw new ISE("Unsupported method in default query metrics implementation.");
}

@Override
public void hasFilters(SearchQuery query)
{
throw new ISE("Unsupported method in default query metrics implementation.");
}

@Override
public void duration(SearchQuery query)
{
throw new ISE("Unsupported method in default query metrics implementation.");
}

@Override
public void queryId(SearchQuery query)
{
throw new ISE("Unsupported method in default query metrics implementation.");
}

@Override
public void granularity(SearchQuery query)
{
// Don't emit by default
}

@Override
public void context(SearchQuery query)
{
delegateQueryMetrics.context(query);
}

@Override
public void server(String host)
{
delegateQueryMetrics.server(host);
}

@Override
public void remoteAddress(String remoteAddress)
{
delegateQueryMetrics.remoteAddress(remoteAddress);
}

@Override
public void status(String status)
{
delegateQueryMetrics.status(status);
}

@Override
public void success(boolean success)
{
delegateQueryMetrics.success(success);
}

@Override
public void segment(String segmentIdentifier)
{
delegateQueryMetrics.segment(segmentIdentifier);
}

@Override
public void chunkInterval(Interval interval)
{
delegateQueryMetrics.chunkInterval(interval);
}

@Override
public void preFilters(List<Filter> preFilters)
{
delegateQueryMetrics.preFilters(preFilters);
}

@Override
public void postFilters(List<Filter> postFilters)
{
delegateQueryMetrics.postFilters(postFilters);
}

@Override
public BitmapResultFactory<?> makeBitmapResultFactory(BitmapFactory factory)
{
return delegateQueryMetrics.makeBitmapResultFactory(factory);
}

@Override
public QueryMetrics reportQueryTime(long timeNs)
{
return delegateQueryMetrics.reportQueryTime(timeNs);
}

@Override
public QueryMetrics reportQueryBytes(long byteCount)
{
return delegateQueryMetrics.reportQueryBytes(byteCount);
}

@Override
public QueryMetrics reportWaitTime(long timeNs)
{
return delegateQueryMetrics.reportWaitTime(timeNs);
}

@Override
public QueryMetrics reportSegmentTime(long timeNs)
{
return delegateQueryMetrics.reportSegmentTime(timeNs);
}

@Override
public QueryMetrics reportSegmentAndCacheTime(long timeNs)
{
return delegateQueryMetrics.reportSegmentAndCacheTime(timeNs);
}

@Override
public QueryMetrics reportIntervalChunkTime(long timeNs)
{
return delegateQueryMetrics.reportIntervalChunkTime(timeNs);
}

@Override
public QueryMetrics reportCpuTime(long timeNs)
{
return delegateQueryMetrics.reportCpuTime(timeNs);
}

@Override
public QueryMetrics reportNodeTimeToFirstByte(long timeNs)
{
return delegateQueryMetrics.reportNodeTimeToFirstByte(timeNs);
}

@Override
public QueryMetrics reportNodeTime(long timeNs)
{
return delegateQueryMetrics.reportNodeTime(timeNs);
}

@Override
public QueryMetrics reportNodeBytes(long byteCount)
{
return delegateQueryMetrics.reportNodeBytes(byteCount);
}

@Override
public QueryMetrics reportBitmapConstructionTime(long timeNs)
{
return delegateQueryMetrics.reportBitmapConstructionTime(timeNs);
}

@Override
public QueryMetrics reportSegmentRows(long numRows)
{
return delegateQueryMetrics.reportSegmentRows(numRows);
}

@Override
public QueryMetrics reportPreFilteredRows(long numRows)
{
return delegateQueryMetrics.reportPreFilteredRows(numRows);
}

@Override
public void emit(ServiceEmitter emitter)
{
delegateQueryMetrics.emit(emitter);
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
/*
* Licensed to Metamarkets Group Inc. (Metamarkets) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. Metamarkets licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/

package io.druid.query.search;

import com.google.common.annotations.VisibleForTesting;
import com.google.inject.Inject;
import io.druid.query.DefaultGenericQueryMetricsFactory;
import io.druid.query.GenericQueryMetricsFactory;

public class DefaultSearchQueryMetricsFactory implements SearchQueryMetricsFactory
{
private static final SearchQueryMetricsFactory INSTANCE =
new DefaultSearchQueryMetricsFactory(DefaultGenericQueryMetricsFactory.instance());
private final GenericQueryMetricsFactory genericQueryMetricsFactory;

@Inject
public DefaultSearchQueryMetricsFactory(GenericQueryMetricsFactory genericQueryMetricsFactory)
{
this.genericQueryMetricsFactory = genericQueryMetricsFactory;
}

@VisibleForTesting
public static SearchQueryMetricsFactory instance()
{
return INSTANCE;
}

@Override
public SearchQueryMetrics makeMetrics(SearchQuery query)
{
return new DefaultSearchQueryMetrics(genericQueryMetricsFactory.makeMetrics(query));
}
}
Loading