Faster JSON marshaling (and easyjson support)

[jacksontj]

I've been looking into options for fixing #3601 (JSON performance issues). I see that there is an open PR related to this issue #3536 -- which simply switches to jsoniter.

From my testing there are effectively 2 issues.

First issue
There are a few places in the marshaling process that require making lots of copies of the underlying datapoints. This not only causes memory bloat, but this significantly slows down the process. Similarly the task is made unecessarily difficult by converting int64 to float64 before putting it to a string (in the case of Time).

Second Issue
The majority of the performance issues at "large" (quoted, because large is only hours of data) are due to the large memory requirements due to marshaling the entire structure into memory before returning it. easyjson has a streaming serialization interface which solves this problem. When using the encoding/json library to do the marshaling I see my responses regularly increase by ~6x in memory before being written out. With this streaming approach I see no such memory increase.

Of course any performance change is only as good as the benchmarks associated with it (which are included in this PR). On my X1 carbon 5th gen, here are the results:

Before these changes

$ go test -run=x -bench=BenchmarkMarshal
goos: linux
goarch: amd64
pkg: github.com/prometheus/common/model
BenchmarkMarshal/SampleValue/encoding/json-4         	 2000000	       617 ns/op
model.SampleValue not easyjson.Marshaler
BenchmarkMarshal/Timestamp/encoding/json-4           	 2000000	       598 ns/op
model.Time not easyjson.Marshaler
BenchmarkMarshal/SamplePair/encoding/json-4          	 1000000	      2069 ns/op
model.SamplePair not easyjson.Marshaler
BenchmarkMarshal/labelset/encoding/json-4            	 2000000	       686 ns/op
model.Metric not easyjson.Marshaler
BenchmarkMarshal/SampleStream/encoding/json-4        	   10000	    198699 ns/op
*model.SampleStream not easyjson.Marshaler
BenchmarkMarshal/Matrix/encoding/json-4              	      20	  96541250 ns/op
model.Matrix not easyjson.Marshaler
PASS
ok  	github.com/prometheus/common/model	11.913s

After these changes

$ go test -run=x -bench=BenchmarkMarshal
goos: linux
goarch: amd64
pkg: github.com/prometheus/common/model
BenchmarkMarshal/SampleValue/encoding/json-4         	 3000000	       516 ns/op
BenchmarkMarshal/SampleValue/easyjson-4              	10000000	       209 ns/op
BenchmarkMarshal/Timestamp/encoding/json-4           	 3000000	       518 ns/op
BenchmarkMarshal/Timestamp/easyjson-4                	10000000	       211 ns/op
BenchmarkMarshal/SamplePair/encoding/json-4          	 2000000	       719 ns/op
BenchmarkMarshal/SamplePair/easyjson-4               	 5000000	       305 ns/op
BenchmarkMarshal/labelset/encoding/json-4            	 2000000	       663 ns/op
BenchmarkMarshal/labelset/easyjson-4                 	10000000	       214 ns/op
BenchmarkMarshal/SampleStream/encoding/json-4        	   50000	     30999 ns/op
BenchmarkMarshal/SampleStream/easyjson-4             	  100000	     16747 ns/op
BenchmarkMarshal/Matrix/encoding/json-4              	     100	  14360224 ns/op
BenchmarkMarshal/Matrix/easyjson-4                   	     200	   8192013 ns/op
PASS
ok  	github.com/prometheus/common/model	25.163s

For ease of reading, here is the same data in a table:

	encoding/json (before)	encoding/json (after	encoding/json (speedup)	easyjson	easyjson (speedup)
SampleValue	617	516	16%	209	66%
Timestamp	598	518	13%	211	64%
SamplePair	2069	719	65%	305	85%
labelset	686	663	3%	214	68%
SampleStream	198699	30999	84%	16747	91%
Matrix	96541250	14360224	85%	8192013	91%

[knweiss]

@jacksontj FWIW: You can use benchcmp or benchstat to compare and print both benchmark results side by side.

[jacksontj]

@knweiss thanks, here is the output from benchcmp:

benchmark                                         old ns/op     new ns/op     delta
BenchmarkMarshal/SampleValue/encoding/json-4      617           516           -16.37%
BenchmarkMarshal/Timestamp/encoding/json-4        598           518           -13.38%
BenchmarkMarshal/SamplePair/encoding/json-4       2069          719           -65.25%
BenchmarkMarshal/labelset/encoding/json-4         686           663           -3.35%
BenchmarkMarshal/SampleStream/encoding/json-4     198699        30999         -84.40%
BenchmarkMarshal/Matrix/encoding/json-4           96541250      14360224      -85.13%

[jacksontj]

Benmark comparison run again with memory numbers included:

benchmark                                         old ns/op     new ns/op     delta
BenchmarkMarshal/SampleValue/encoding/json-4      766           468           -38.90%
BenchmarkMarshal/Timestamp/encoding/json-4        713           586           -17.81%
BenchmarkMarshal/SamplePair/encoding/json-4       2050          743           -63.76%
BenchmarkMarshal/labelset/encoding/json-4         683           662           -3.07%
BenchmarkMarshal/SampleStream/encoding/json-4     197673        19165         -90.30%
BenchmarkMarshal/Matrix/encoding/json-4           96928336      8494863       -91.24%

benchmark                                         old allocs     new allocs     delta
BenchmarkMarshal/SampleValue/encoding/json-4      6              3              -50.00%
BenchmarkMarshal/Timestamp/encoding/json-4        5              4              -20.00%
BenchmarkMarshal/SamplePair/encoding/json-4       20             5              -75.00%
BenchmarkMarshal/labelset/encoding/json-4         7              7              +0.00%
BenchmarkMarshal/SampleStream/encoding/json-4     2012           118            -94.14%
BenchmarkMarshal/Matrix/encoding/json-4           1003160        50079          -95.01%

benchmark                                         old bytes     new bytes     delta
BenchmarkMarshal/SampleValue/encoding/json-4      497           400           -19.52%
BenchmarkMarshal/Timestamp/encoding/json-4        336           416           +23.81%
BenchmarkMarshal/SamplePair/encoding/json-4       1264          424           -66.46%
BenchmarkMarshal/labelset/encoding/json-4         416           416           +0.00%
BenchmarkMarshal/SampleStream/encoding/json-4     116137        5035          -95.66%
BenchmarkMarshal/Matrix/encoding/json-4           58131741      2150175       -96.30%

[jacksontj]

cc @fabxc @juliusv @beorn7

[juliusv]

nit: Since maps are reference types and we're not mutating the labelset, is the pointer receiver useful here? A value receiver would allow just saying range l instead of range *l below.

[jacksontj]

@juliusv looks like it won't matter, I was worried about copies etc. but it seems that the compiler is smart enough-- changed.

[juliusv]

nit: be consistent how you handle the if-else here and in the LabelSet marshaling method (I guess I prefer the if-else variant, since it only assigns the first variable once - not that I think it'd make a real difference).

[jacksontj]

Good catch, I switched to the else, since it seems better :) (tests don't show much of a difference, but it looks better at least)

[brian-brazil]

I don't think this is an appropriate dependency to have in this package. Data models should just be data models.

[jacksontj]

@brian-brazil this package already has a dep on encoding/json meaning marshaling is already a part of the model. Similarly custom marshal and unmarshal methods already exist in this package (and even in that file).

In golang you can't define methods on a struct outside of the package where the struct is defined, meaning that if you want a struct to adhere to an interface it must define the methods in the same package. We could move the marshal methods into a different file in the same package, but the existing ones are in the same files-- but if separate files in the package is preferred I'd be more than happy to move them around.

[jacksontj]

@juliusv Updated, I squashed the commit so the merge will be nicer :)

[bboreham]

You might want to take a look at #114 - a combination of the two techniques might work best.

Results from the microbenchmarks in #114 on my desktop machine:

BenchmarkMarshalJSON-3     	20000000	       110 ns/op	      16 B/op	       1 allocs/op
BenchmarkUnmarshalJSON-3   	20000000	        85.5 ns/op	      16 B/op	       2 allocs/op

and same tests against this PR:

BenchmarkMarshalJSON-3     	10000000	       167 ns/op	     144 B/op	       2 allocs/op
BenchmarkUnmarshalJSON-3   	10000000	       151 ns/op	      48 B/op	       2 allocs/op

[jacksontj]

@bboreham thanks for the ping! I actually have a separate branch where I've added faster marshal/unmarshal for most of the structs in common. I'm running a fork of prometheus to test out the performance gains and they are significant (>50% CPU reduction, >3x latency improvement, etc.) If this PR gets some traction I'll update it with those (if you are interested you can see my changes -- master...jacksontj:tjackson_fork)

[brian-brazil]

This code isn't even being used by the Prometheus web API, and we've gone with a different json encoder.