Graphite emitter

docs/content/configuration/index.md

@@ -150,6 +150,11 @@ The Druid servers emit various metrics and alerts via something we call an Emitt
 |--------|-----------|-------|
 |`druid.emitter.composing.emitters`|List of emitter modules to load e.g. ["logging","http"].|[]|
 
+#### Graphite Emitter
+
+To use graphite as emitter set `druid.emitter=graphite`. For configuration details please follow this [link](https://github.com/druid-io/druid/tree/master/extensions/graphite-emitter/README.md).
+
+
 ### Metadata Storage
 
 These properties specify the jdbc connection and other configuration around the metadata storage. The only processes that connect to the metadata storage with these properties are the [Coordinator](../design/coordinator.html), [Indexing service](../design/indexing-service.html) and [Realtime Nodes](../design/realtime.html).

extensions/graphite-emitter/README.md

@@ -0,0 +1,72 @@
+## introduction
+
+This extension emits druid metrics to a graphite carbon server.
+Events are sent after been [pickled](http://graphite.readthedocs.org/en/latest/feeding-carbon.html#the-pickle-protocol); the size of the batch is configurable. 
+
+## configuration
+
+All the configuration parameters for graphite emitter are under `druid.emitter.graphite`.
+
+|property|description|required?|default|
+|--------|-----------|---------|-------|
+|`druid.emitter.graphite.hostname`|The hostname of the graphite server.|yes|none|
+|`druid.emitter.graphite.port`|The port of the graphite server.|yes|none|
+|`druid.emitter.graphite.batchSize`|Number of events to send as one batch.|no|100|
+|`druid.emitter.graphite.eventConverter`| Filter and converter of druid events to graphite event(please see next section). |yes|none|  
+|`druid.emitter.graphite.flushPeriod` | Queue flushing period in milliseconds. |no|1 minute|
+|`druid.emitter.graphite.maxQueueSize`| Maximum size of the queue used to buffer events. |no|`MAX_INT`|
+|`druid.emitter.graphite.alertEmitters`| List of emitters where alerts will be forwarded to. |no| empty list (no forwarding)|
+
+### Druid to Graphite Event Converter
+
+Graphite Event Converter defines a mapping between druid metrics name plus dimensions to a Graphite metric path.
+Graphite metric path is organized using the following schema:
+`<namespacePrefix>.[<druid service name>].[<druid hostname>].<druid metrics dimensions>.<druid metrics name>`
+Properly naming the metrics is critical to avoid conflicts, confusing data and potentially wrong interpretation later on.
+
+Example `druid.historical.hist-host1_yahoo_com:8080.MyDataSourceName.GroupBy.query/time`:
+
+ * `druid` -> namespace prefix 
+ * `historical` -> service name 
+ * `hist-host1.yahoo.com:8080` -> druid hostname
+ * `MyDataSourceName` -> dimension value 
+ * `GroupBy` -> dimension value
+ * `query/time` -> metric name
+
+We have two different implementation of event converter:
+
+#### Send-All converter
+
+The first implementation called `all`, will send all the druid service metrics events. 
+The path will be in the form `<namespacePrefix>.[<druid service name>].[<druid hostname>].<dimensions values ordered by dimension's name>.<metric>`
+User has control of `<namespacePrefix>.[<druid service name>].[<druid hostname>].`
+
+You can omit the hostname by setting `ignoreHostname=true`
+`druid.SERVICE_NAME.dataSourceName.queryType.query.time`
+
+You can omit the service name by setting `ignoreServiceName=true`
+`druid.HOSTNAME.dataSourceName.queryType.query.time`
+
+```json
+
+druid.emitter.graphite.eventConverter={"type":"all", "namespacePrefix": "druid.test", "ignoreHostname":true, "ignoreServiceName":true}
+
+```
+
+#### White-list based converter
+
+The second implementation called `whiteList`, will send only the white listed metrics and dimensions.
+Same as for the `all` converter user has control of `<namespacePrefix>.[<druid service name>].[<druid hostname>].`
+White-list based converter comes with the following  default white list map located under resources [defaultWhiteListMap.json](./src/main/resources/defaultWhiteListMap.json)
+
+Although user can override the default white list map by supplying a property called `mapPath`.
+This property is a String containing  the path for the file containing **white list map Json object**.
+For example the following converter will read the map from the file `/pathPrefix/fileName.json`.  
+
+```json
+
+druid.emitter.graphite.eventConverter={"type":"whiteList", "namespacePrefix": "druid.test", "ignoreHostname":true, "ignoreServiceName":true, "mapPath":"/pathPrefix/fileName.json"}
+
+```
+
+**Druid emits a huge number of metrics we highly recommend to use the `whiteList` converter**

extensions/graphite-emitter/pom.xml

@@ -0,0 +1,91 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+  ~ Licensed to Metamarkets Group Inc. (Metamarkets) under one
+  ~ or more contributor license agreements. See the NOTICE file
+  ~ distributed with this work for additional information
+  ~ regarding copyright ownership. Metamarkets licenses this file
+  ~ to you under the Apache License, Version 2.0 (the
+  ~ "License"); you may not use this file except in compliance
+  ~ with the License. You may obtain a copy of the License at
+  ~
+  ~ http://www.apache.org/licenses/LICENSE-2.0
+  ~
+  ~ Unless required by applicable law or agreed to in writing,
+  ~ software distributed under the License is distributed on an
+  ~ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+  ~ KIND, either express or implied. See the License for the
+  ~ specific language governing permissions and limitations
+  ~ under the License.
+-->
+
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
+  <modelVersion>4.0.0</modelVersion>
+
+  <parent>
+    <groupId>io.druid</groupId>
+    <artifactId>druid</artifactId>
+    <version>0.9.0-SNAPSHOT</version>
+    <relativePath>../../pom.xml</relativePath>
+  </parent>
+
+  <groupId>io.druid.extensions</groupId>
+  <artifactId>graphite-emitter</artifactId>
+  <name>graphite-emitter</name>
+  <description>Druid emitter extension to support graphite</description>
+
+  <dependencies>
+    <dependency>
+      <groupId>io.druid</groupId>
+      <artifactId>druid-common</artifactId>
+      <version>${project.parent.version}</version>
+      <scope>provided</scope>
+    </dependency>
+    <dependency>
+      <groupId>io.druid</groupId>
+      <artifactId>druid-api</artifactId>
+      <version>${druid.api.version}</version>
+      <scope>provided</scope>
+    </dependency>
+    <dependency>
+      <groupId>com.metamx</groupId>
+      <artifactId>emitter</artifactId>
+      <scope>provided</scope>
+    </dependency>
+    <dependency>
+      <groupId>io.dropwizard.metrics</groupId>
+      <artifactId>metrics-graphite</artifactId>
+      <version>3.1.2</version>
+    </dependency>
+    <dependency>
+      <groupId>junit</groupId>
+      <artifactId>junit</artifactId>
+      <scope>test</scope>
+    </dependency>
+    <dependency>
+      <groupId>org.easymock</groupId>
+      <artifactId>easymock</artifactId>
+      <scope>test</scope>
+    </dependency>
+    <dependency>
+      <groupId>pl.pragmatists</groupId>
+      <artifactId>JUnitParams</artifactId>
+      <version>1.0.4</version>
+      <scope>test</scope>
+    </dependency>
+    <dependency>
+      <groupId>io.druid</groupId>
+      <artifactId>druid-server</artifactId>
+      <version>${project.parent.version}</version>
+      <type>test-jar</type>
+      <scope>test</scope>
+    </dependency>
+    <dependency>
+      <groupId>io.druid</groupId>
+      <artifactId>druid-processing</artifactId>
+      <version>${project.parent.version}</version>
+      <type>test-jar</type>
+      <scope>test</scope>
+    </dependency>
+  </dependencies>
+</project>

extensions/graphite-emitter/src/main/java/io/druid/emitter/graphite/DruidToGraphiteEventConverter.java

@@ -0,0 +1,44 @@
+/*
+ *  Licensed to Metamarkets Group Inc. (Metamarkets) under one
+ *  or more contributor license agreements. See the NOTICE file
+ *  distributed with this work for additional information
+ *  regarding copyright ownership. Metamarkets licenses this file
+ *  to you under the Apache License, Version 2.0 (the
+ *  "License"); you may not use this file except in compliance
+ *  with the License. You may obtain a copy of the License at
+ *
+ *  http://www.apache.org/licenses/LICENSE-2.0
+ *
+ *  Unless required by applicable law or agreed to in writing,
+ *  software distributed under the License is distributed on an
+ *  "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ *  KIND, either express or implied. See the License for the
+ *  specific language governing permissions and limitations
+ *  under the License.
+ */
+
+package io.druid.emitter.graphite;
+
+import com.fasterxml.jackson.annotation.JsonSubTypes;
+import com.fasterxml.jackson.annotation.JsonTypeInfo;
+import com.metamx.emitter.service.ServiceMetricEvent;
+
+
+@JsonTypeInfo(use = JsonTypeInfo.Id.NAME, property = "type", defaultImpl = WhiteListBasedConverter.class)
+@JsonSubTypes(value= {
+    @JsonSubTypes.Type(name="all", value = SendAllGraphiteEventConverter.class),
+    @JsonSubTypes.Type(name="whiteList", value = WhiteListBasedConverter.class)
+})
+
+public interface DruidToGraphiteEventConverter
+{
+  /**
+   * This function acts as a filter. It returns <tt>null</tt> if the event is not suppose to be emitted to Graphite
+   * Also This function will define the mapping between the druid event dimension's values and Graphite metric Path
+   *
+   * @param serviceMetricEvent Druid event ot type {@link ServiceMetricEvent}
+   *
+   * @return {@link GraphiteEvent} or <tt>null</tt>
+   */
+  GraphiteEvent druidEventToGraphite(ServiceMetricEvent serviceMetricEvent);
+}