Add helper util to convert OC Metrics data models to SD monitoring data models by mayurkale22 · Pull Request #266 · census-instrumentation/opencensus-node

mayurkale22 · 2019-01-09T02:12:40Z

This PR is part of #257, Basically, StackdriverStatsExporterUtils methods will be used by StackdriverStatsExporter to translate data from OpenCensus data models to StackDriver monitoring data models. I will send separate PR to include this util in StackdriverStatsExporter.

mayurkale22 · 2019-01-09T02:14:47Z

packages/opencensus-exporter-stackdriver/src/types.ts


 export interface Point {
-  interval: {endTime: string, startTime: string};
+  interval: {endTime: string, startTime?: string};


startTime should be optional (Must be present for cumulative metrics only). More info: https://cloud-dot-devsite.googleplex.com/monitoring/api/ref_v3/rest/v3/TimeInterval

mayurkale22 · 2019-01-09T02:18:28Z

packages/opencensus-exporter-stackdriver/src/types.ts

 export interface TimeSeries {
  metric: {type: string; labels: {[key: string]: string};};
-  resource: {type: 'global', labels: {[key: string]: string}};
+  resource: {type: string, labels: {[key: string]: string}};


In the future, we have planned to add more type like k8s_container, gce_instance and aws_ec2_instance.

songy23

LGTM overall

songy23 · 2019-01-09T03:35:55Z

packages/opencensus-exporter-stackdriver/src/stackdriver-stats-utils.ts

+      timeSeriesList.push({
+        metric: this.createMetric(
+            metricDescriptor, timeSeries.labelValues, metricPrefix),
+        resource: monitoredResource,


Just to double check: it's fine to use the same monitoredResource here (i.e no need to convert)?

Currently no, but once we have monitored resource util in place (#173) we need to do the conversion same as Java's implementation.

draffensperger · 2019-01-09T14:07:58Z

packages/opencensus-exporter-stackdriver/src/stackdriver-stats-utils.ts

+ * monitoring data models.
+ */
+export class StackdriverStatsExporterUtils {
+  static readonly OPENCENSUS_TASK: string = 'opencensus_task';


Why are these constants and methods static within a class? While that is needed in Java, IMO it's just extra wrapping for TypeScript since the module itself already acts as a namespace. What would you think about just exporting the functions and constants directly?

draffensperger · 2019-01-09T14:09:24Z

packages/opencensus-exporter-stackdriver/src/stackdriver-stats-utils.ts

+      valueType: this.createValueType(metricDescriptor.type),
+      unit: metricDescriptor.unit,
+      labels: this.createLabelDescriptor(metricDescriptor.labelKeys)
+    } as MetricDescriptor;


Could you remove the as MetricDescriptor? The TS compiler should know the object must match the type based on the method return type.

draffensperger · 2019-01-09T14:10:01Z

packages/opencensus-exporter-stackdriver/src/stackdriver-stats-utils.ts

+    const metricKind = this.createMetricKind(metricDescriptor.type);
+    const valueType = this.createValueType(metricDescriptor.type);
+
+    for (const timeSeries of metric.timeseries) {


Optional: could we use map here instead of a for loop + push?

draffensperger · 2019-01-09T14:17:24Z

packages/opencensus-exporter-stackdriver/src/stackdriver-stats-utils.ts

+   */
+  static createLabelDescriptor(labelKeys: LabelKey[]): LabelDescriptor[] {
+    const labelDescriptorList = labelKeys.map(labelKey => {
+      return {


Optional, you can avoid the return if you put the object in paratheses, so labelKeys.map(labelKey => ({ key: ...}))

draffensperger · 2019-01-09T14:22:37Z

packages/opencensus-exporter-stackdriver/src/stackdriver-stats-utils.ts

+        key: labelKey.key,
+        valueType: 'STRING',  // Now we only support String type.
+        description: labelKey.description
+      } as LabelDescriptor;


Could you use a type annotation on labelDescriptorList, i.e. const labelDesceriptorList: LabelDescriptor[] = rather than a type assertion as LabelDescriptor here? The type annotation will cause the code to fail to compile if the interface changes so would help detect refactoring bugs.

draffensperger · 2019-01-09T14:23:52Z

packages/opencensus-exporter-stackdriver/src/stackdriver-stats-utils.ts

+   */
+  static createMetric(
+      metricDescriptor: OCMetricDescriptor, labelValues: LabelValue[],
+      metricPrefix: string): {type: string; labels: {[key: string]: string};} {


The {type: string; labels: {[key: string]: string};} type looks fairly complex, what would you think about making it a first class exported type alias?

draffensperger · 2019-01-09T14:25:27Z

packages/opencensus-exporter-stackdriver/src/stackdriver-stats-utils.ts

+  static createPoint(
+      point: TimeSeriesPoint, startTimeStamp: Timestamp,
+      valueType: ValueType): Point {
+    let value;


Optional: you can avoid use of let here by creating a separate function like createValue or similar that would take in the value type and value and return the value, then just have const value = createValue(valueType, point)

draffensperger · 2019-01-09T14:26:08Z

packages/opencensus-exporter-stackdriver/src/stackdriver-stats-utils.ts

+            this.createDistribution(point.value as DistributionValue)
+      };
+    } else {
+      // console.log(`${valueType} is not supported.`);


Could you remove this commented code and else black?

draffensperger · 2019-01-09T14:28:56Z

packages/opencensus-exporter-stackdriver/src/stackdriver-stats-utils.ts

+    return str.replace('000Z', `${nsStr}Z`);
+  }
+
+  private static padNS(ns: number) {


Could you make this function name a bit more descriptive? It seems like it's left padding a number, but I wouldn't know that without reading the code.

mayurkale22 · 2019-01-09T20:23:11Z

@draffensperger thanks for the reviews. I have addressed all the comments, PTAL.

draffensperger

Thanks for the changes!

justindsmith

Overall looks good, just one small log removal

justindsmith · 2019-01-10T05:47:28Z

packages/opencensus-exporter-stackdriver/src/stackdriver-stats-utils.ts

+
+/** Returns a task label value in the format of 'nodejs-<pid>@<hostname>'. */
+function generateDefaultTaskValue(): string {
+  console.log('inside generateDefaultTaskValue');


Remove console.log

… data models

codecov-io · 2019-01-10T06:55:39Z

Codecov Report

Merging #266 into master will increase coverage by 3.3%.
The diff coverage is n/a.

@@           Coverage Diff            @@
##           master    #266     +/-   ##
========================================
+ Coverage    91.5%   94.8%   +3.3%     
========================================
  Files         108     110      +2     
  Lines        7520    7779    +259     
  Branches      692     713     +21     
========================================
+ Hits         6881    7375    +494     
+ Misses        639     404    -235

Impacted Files	Coverage Δ
test/test-stackdriver-stats-utils.ts	`100% <0%> (ø)`
src/stackdriver-stats-utils.ts	`99.03% <0%> (ø)`
src/http.ts	`89.8% <0%> (+0.97%)`	⬆️
src/http2.ts	`89.83% <0%> (+82.2%)`	⬆️
test/test-http2.ts	`98.66% <0%> (+91.33%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 47a672e...4c4a190. Read the comment docs.

googlebot added the cla: yes label Jan 9, 2019

mayurkale22 added exporters Metrics labels Jan 9, 2019

mayurkale22 commented Jan 9, 2019

View reviewed changes

mayurkale22 requested review from draffensperger, justindsmith and songy23 January 9, 2019 02:18

songy23 reviewed Jan 9, 2019

View reviewed changes

draffensperger reviewed Jan 9, 2019

View reviewed changes

mayurkale22 force-pushed the stackdriver-stats-utils branch 3 times, most recently from de638fb to eeaf2a2 Compare January 9, 2019 20:21

draffensperger approved these changes Jan 9, 2019

View reviewed changes

songy23 approved these changes Jan 9, 2019

View reviewed changes

justindsmith approved these changes Jan 10, 2019

View reviewed changes

mayurkale22 added 3 commits January 9, 2019 22:52

Add helper util for coverting OC Metrics data models to SD monitoring…

83df2c8

… data models

fix review comments

c9aa60a

remove temp log

4c4a190

mayurkale22 force-pushed the stackdriver-stats-utils branch from 09a70f3 to 4c4a190 Compare January 10, 2019 06:52

mayurkale22 merged commit 3db8bf5 into census-instrumentation:master Jan 10, 2019

mayurkale22 deleted the stackdriver-stats-utils branch January 10, 2019 06:56

mayurkale22 mentioned this pull request Jan 11, 2019

Stats/Stackdriver: export data using metrics API #275

Merged

mayurkale22 mentioned this pull request Feb 12, 2019

chore(multiple): 0.0.9 release proposal #341

Merged

Conversation

mayurkale22 commented Jan 9, 2019

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

songy23 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mayurkale22 commented Jan 9, 2019

Uh oh!

draffensperger left a comment

Choose a reason for hiding this comment

Uh oh!

justindsmith left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

codecov-io commented Jan 10, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

codecov-io commented Jan 10, 2019 •

edited

Loading