Skip to content

Improves benchmark comparison#77

Merged
springmeyer merged 3 commits intomasterfrom
bench-improvements
Jul 22, 2018
Merged

Improves benchmark comparison#77
springmeyer merged 3 commits intomasterfrom
bench-improvements

Conversation

@springmeyer
Copy link
Contributor

@springmeyer springmeyer commented Jul 22, 2018

This fixes a few issues in / improves the benchmark script:

  • Option to test compositing from compressed tiles is called --compress but usage said --compressed
  • vtcomposite mode was not tracking memory and memory stats were not output at all
  • We were using vt.addDataSync for node-mapnik, whereas in production we use the async vt.addData due to https://github.com/mapbox/tilelive-vector/blob/master/backend.js#L113 (which means that gzip decompression is async)
  • refactors code to avoid duplication between vtcomposite and node-mapnik modes

/cc @millzpaugh @artemp

@springmeyer
Copy link
Contributor Author

springmeyer commented Jul 22, 2018

Looks like the async node-mapnik approach is significantly faster (in some cases):

node bench/bench.js --iterations 500 --concurrency 4 --package node-mapnik --compress
1: single tile in/out ... 2392 runs/s (209ms)
2: two different tiles at the same zoom level, zero buffer ... 484 runs/s (1034ms)
3: two different tiles from different zoom levels (separated by one zoom), zero buffer ... 209 runs/s (2391ms)
4: two different tiles from different zoom levels (separated by more than one zoom), zero buffer ... 310 runs/s (1612ms)
5: tiles completely made of points, overzooming, no properties ... 3788 runs/s (132ms)
6: tiles completely made of points, same zoom, no properties ... 15152 runs/s (33ms)
7: tiles completely made of points, overzoooming, lots of properties ... 1276 runs/s (392ms)
8: tiles completely made of points, same zoom, lots of properties ... 10417 runs/s (48ms)
9: buffer_size 128 - tiles completely made of points, same zoom, lots of properties ... 9804 runs/s (51ms)
10: tiles completely made of linestrings, overzooming and lots of properties ... 480 runs/s (1042ms)
11: tiles completely made of polygons, overzooming and lots of properties ... 161 runs/s (3106ms)
12: tiles completely made of points and linestrings, overzooming and lots of properties ... 3937 runs/s (127ms)
13: returns compressed buffer - tiles completely made of points and linestrings, overzooming and lots of properties ... 3937 runs/s (127ms)
14: buffer_size 128 - tiles completely made of points and linestrings, overzooming and lots of properties ... 4065 runs/s (123ms)
15: tiles completely made of points and linestrings, overzooming (2x) and lots of properties ... 5882 runs/s (85ms)
16: tiles completely made of polygons, overzooming and lots of properties ... 442 runs/s (1132ms)
17: tiles completely made of polygons, overzooming (2x) and lots of properties ... 852 runs/s (587ms)
18: return compressed buffer - tiles completely made of polygons, overzooming (2x) and lots of properties ... 845 runs/s (592ms)
19: buffer_size 4096 - tiles completely made of polygons, overzooming (2x) and lots of properties ... 833 runs/s (600ms)

Against master/sync addData:

$ node bench/bench.js --iterations 500 --concurrency 4 --package node-mapnik --compress

1: single tile in/out ... 1374 runs/s (364ms)
2: two different tiles at the same zoom level, zero buffer ... 237 runs/s (2106ms)
3: two different tiles from different zoom levels (separated by one zoom), zero buffer ... 204 runs/s (2452ms)
4: two different tiles from different zoom levels (separated by more than one zoom), zero buffer ... 239 runs/s (2096ms)
5: tiles completely made of points, overzooming, no properties ... 3968 runs/s (126ms)
6: tiles completely made of points, same zoom, no properties ... 7463 runs/s (67ms)
7: tiles completely made of points, overzoooming, lots of properties ... 1232 runs/s (406ms)
8: tiles completely made of points, same zoom, lots of properties ... 4310 runs/s (116ms)
9: buffer_size 128 - tiles completely made of points, same zoom, lots of properties ... 4167 runs/s (120ms)
10: tiles completely made of linestrings, overzooming and lots of properties ... 469 runs/s (1067ms)
11: tiles completely made of polygons, overzooming and lots of properties ... 157 runs/s (3180ms)
12: tiles completely made of points and linestrings, overzooming and lots of properties ... 4274 runs/s (117ms)
13: returns compressed buffer - tiles completely made of points and linestrings, overzooming and lots of properties ... 3968 runs/s (126ms)
14: buffer_size 128 - tiles completely made of points and linestrings, overzooming and lots of properties ... 4464 runs/s (112ms)
15: tiles completely made of points and linestrings, overzooming (2x) and lots of properties ... 6024 runs/s (83ms)
16: tiles completely made of polygons, overzooming and lots of properties ... 432 runs/s (1158ms)
17: tiles completely made of polygons, overzooming (2x) and lots of properties ... 760 runs/s (658ms)
18: return compressed buffer - tiles completely made of polygons, overzooming (2x) and lots of properties ... 839 runs/s (596ms)
19: buffer_size 4096 - tiles completely made of polygons, overzooming (2x) and lots of properties ... 852 runs/s (587ms)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant