Improve performance2: reduce unneeded allocation #16

haya14busa · 2016-09-19T18:27:20Z

ref: #4

…rovement $ go test -v -run="^$" -bench="." -benchtime=5s -cpuprofile=prof.cpu -memprofile=prof.mem | tee prof.0 BenchmarkParseFile-4 30 245095530 ns/op $ go tool pprof --alloc_space go-vimlparser.test prof.mem Entering interactive mode (type "help" for commands) (pprof) top 1401.64MB of 1678.59MB total (83.50%) Dropped 27 nodes (cum <= 8.39MB) Showing top 10 nodes out of 56 (cum >= 143.54MB) flat flat% sum% cum cum% 297.09MB 17.70% 17.70% 345.10MB 20.56% github.com/haya14busa/go-vimlparser/go.NewExportNode 254.71MB 15.17% 32.87% 254.71MB 15.17% github.com/haya14busa/go-vimlparser/go.(*StringReader).__init__ 195.13MB 11.62% 44.50% 352.14MB 20.98% github.com/haya14busa/go-vimlparser/go.(*ExprTokenizer).get 193.57MB 11.53% 56.03% 468.70MB 27.92% github.com/haya14busa/go-vimlparser/go.(*ExprParser).parse_expr9 99.08MB 5.90% 61.93% 166.59MB 9.92% github.com/haya14busa/go-vimlparser/internal/exporter.NewNode 83MB 4.94% 66.88% 83MB 4.94% reflect.unsafe_New 75.52MB 4.50% 71.38% 78.02MB 4.65% github.com/haya14busa/go-vimlparser/go.(*ExprParser).parse_curly_parts 71.52MB 4.26% 75.64% 77.02MB 4.59% github.com/haya14busa/go-vimlparser/go.(*ExprParser).parse_dot 66.50MB 3.96% 79.60% 74MB 4.41% github.com/haya14busa/go-vimlparser/go.(*ExprTokenizer).get2 65.52MB 3.90% 83.50% 143.54MB 8.55% github.com/haya14busa/go-vimlparser/go.(*ExprParser).parse_identifier (pprof) list NewExportNode Total: 1.64GB ROUTINE ======================== github.com/haya14busa/go-vimlparser/go.NewExportNode in /home/haya14busa/src/github.com/haya14busa/go-vimlparser/go/export.go 297.09MB 1.10GB (flat, cum) 66.83% of Total . . 132: if n == nil { . . 133: return nil . . 134: } . . 135: list := make([]*ExportNode, 0) . . 136: for _, n := range n.list { . 1.50MB 137: list = append(list, NewExportNode(n)) . . 138: } . . 139: rlist := make([]*ExportNode, 0) . . 140: for _, n := range n.rlist { . 34.51MB 141: rlist = append(rlist, NewExportNode(n)) . . 142: } . . 143: body := make([]*ExportNode, 0) . . 144: for _, n := range n.body { 2.01MB 294.08MB 145: body = append(body, NewExportNode(n)) . . 146: } . . 147: elseif := make([]*ExportNode, 0) . . 148: for _, n := range n.elseif { . 114.53MB 149: elseif = append(elseif, NewExportNode(n)) . . 150: } . . 151: catch := make([]*ExportNode, 0) . . 152: for _, n := range n.catch { . . 153: catch = append(catch, NewExportNode(n)) . . 154: } . . 155: return &ExportNode{ . . 156: Type: n.type_, 18MB 18MB 157: Pos: NewExportPos(n.pos), . 150.54MB 158: Left: NewExportNode(n.left), . 86.53MB 159: Right: NewExportNode(n.right), . 54.02MB 160: Cond: NewExportNode(n.cond), . . 161: Rest: NewExportNode(n.rest), . . 162: List: list, . . 163: Rlist: rlist, . . 164: Body: body, . . 165: Op: n.op, . . 166: Str: n.str, . . 167: Depth: n.depth, . . 168: Value: n.value, . . 169: . 48.01MB 170: Ea: NewExportExArg(n.ea), 6MB 6MB 171: Attr: NewExportFuncAttr(n.attr), . . 172: . 8MB 173: Endfunction: NewExportNode(n.endfunction), . . 174: Elseif: elseif, . 28.01MB 175: Else: NewExportNode(n.else_), . 6.50MB 176: Endif: NewExportNode(n.endif), . 512.16kB 177: Endwhile: NewExportNode(n.endwhile), . . 178: Endfor: NewExportNode(n.endfor), . . 179: Endtry: NewExportNode(n.endtry), . . 180: . . 181: Catch: catch, . . 182: Finally: NewExportNode(n.finally), . . 183: . . 184: Pattern: n.pattern, 271.08MB 271.08MB 185: Curly: n.curly, . . 186: } . . 187:} . . 188: . . 189:type ExportFuncAttr struct { . . 190: Range bool $ go tool pprof go-vimlparser.test prof.cpu Entering interactive mode (type "help" for commands) (pprof) top 4020ms of 9040ms total (44.47%) Dropped 108 nodes (cum <= 45.20ms) Showing top 10 nodes out of 151 (cum >= 420ms) flat flat% sum% cum cum% 1020ms 11.28% 11.28% 1710ms 18.92% runtime.scanobject 750ms 8.30% 19.58% 1430ms 15.82% runtime.mallocgc 330ms 3.65% 23.23% 760ms 8.41% regexp.(*machine).onepass 320ms 3.54% 26.77% 320ms 3.54% sync/atomic.CompareAndSwapUint32 290ms 3.21% 29.98% 290ms 3.21% runtime.heapBitsForObject 290ms 3.21% 33.19% 430ms 4.76% runtime.mapaccess2_faststr 270ms 2.99% 36.17% 270ms 2.99% sync/atomic.AddUint32 260ms 2.88% 39.05% 620ms 6.86% github.com/haya14busa/go-vimlparser/go.NewExportNode 260ms 2.88% 41.92% 540ms 5.97% sync.(*Mutex).Unlock 230ms 2.54% 44.47% 420ms 4.65% runtime.greyobject

$ benchcmp prof.0 prof.1 benchmark old ns/op new ns/op delta BenchmarkParseFile-4 245095530 244779103 -0.13%

$ benchcmp prof.1 prof.2 benchmark old ns/op new ns/op delta BenchmarkParseFile-4 244779103 225677916 -7.80% $ go tool pprof --alloc_space go-vimlparser.test prof.mem Entering interactive mode (type "help" for commands) (pprof) top 1153.55MB of 1363.78MB total (84.58%) Dropped 30 nodes (cum <= 6.82MB) Showing top 10 nodes out of 56 (cum >= 814.41MB) flat flat% sum% cum cum% 261.21MB 19.15% 19.15% 261.21MB 19.15% github.com/haya14busa/go-vimlparser/go.(*StringReader).__init__ 194.56MB 14.27% 33.42% 480.71MB 35.25% github.com/haya14busa/go-vimlparser/go.(*ExprParser).parse_expr9 185.66MB 13.61% 47.03% 348.66MB 25.57% github.com/haya14busa/go-vimlparser/go.(*ExprTokenizer).get 98.50MB 7.22% 54.26% 98.50MB 7.22% reflect.unsafe_New 88.56MB 6.49% 60.75% 112.06MB 8.22% github.com/haya14busa/go-vimlparser/go.NewNode 77.52MB 5.68% 66.43% 84.02MB 6.16% github.com/haya14busa/go-vimlparser/go.(*ExprParser).parse_curly_parts 70.52MB 5.17% 71.60% 154.54MB 11.33% github.com/haya14busa/go-vimlparser/go.(*ExprParser).parse_identifier 69.52MB 5.10% 76.70% 74.02MB 5.43% github.com/haya14busa/go-vimlparser/go.(*ExprParser).parse_dot 59MB 4.33% 81.03% 64.50MB 4.73% github.com/haya14busa/go-vimlparser/go.(*ExprTokenizer).get2 48.51MB 3.56% 84.58% 814.41MB 59.72% github.com/haya14busa/go-vimlparser/go.(*VimLParser).parse_one_cmd

coveralls · 2016-09-19T18:31:26Z

Coverage increased (+2.0%) to 83.751% when pulling 64a3a04 on performance-export into a41844f on master.

haya14busa · 2016-09-19T18:35:00Z

Benchmarks

ref: #4 (comment)

$ pwd
/home/haya14busa/src/github.com/ynkdir/vim-vimlparser

$ git rev-parse HEAD
2fff43c58968a18bc01bc8304df68bde01af04d9

$ wc -l < autoload/vimlparser.vim
5073

$ go get github.com/haya14busa/go-vimlparser/cmd/vimlparser

$ time vimlparser autoload/vimlparser.vim > /dev/null
vimlparser autoload/vimlparser.vim > /dev/null  0.25s user 0.03s system 114% cpu 0.244 total

(autoload/vimlparser.vim is the same as #4 (comment))

Before: 0.36s
After: 0.25s

coveralls · 2016-09-19T18:40:50Z

Coverage increased (+2.0%) to 83.751% when pulling 3a2f2be on performance-export into a41844f on master.

$ benchcmp prof.3 prof.4 benchmark old ns/op new ns/op delta BenchmarkParseFile-4 227103681 221732445 -2.37%

Before: . . 375: . . 376: case NODE_DICT: . . 377: var kvs []ast.KeyValue . . 378: for _, nn := range n.value.([]interface{}) { . . 379: kv := nn.([]interface{}) . 3MB 380: k := newAstNode(kv[0].(*VimNode)) . 2.50MB 381: v := newAstNode(kv[1].(*VimNode)) 5MB 5MB 382: kvs = append(kvs, ast.KeyValue{Key: k, Value: v}) . . 383: } . . 384: return &ast.Dict{ . . 385: Lcurlybrace: pos, . . 386: Entries: kvs, . . 387: } . . 388: After: . . 375: . . 376: case NODE_DICT: . . 377: entries := n.value.([]interface{}) 2MB 2MB 378: kvs := make([]ast.KeyValue, 0, len(entries)) . . 379: for _, nn := range entries { . . 380: kv := nn.([]interface{}) . 3.50MB 381: k := newAstNode(kv[0].(*VimNode)) . 3.50MB 382: v := newAstNode(kv[1].(*VimNode)) . . 383: kvs = append(kvs, ast.KeyValue{Key: k, Value: v}) . . 384: } . . 385: return &ast.Dict{ . . 386: Lcurlybrace: pos, 1MB 1MB 387: Entries: kvs, . . 388: }

coveralls · 2016-09-19T19:06:09Z

Coverage increased (+2.0%) to 83.763% when pulling 47a8076 on performance-export into a41844f on master.

haya14busa added 5 commits September 20, 2016 02:18

slice allocation

1bae2fd

$ benchcmp prof.0 prof.1 benchmark old ns/op new ns/op delta BenchmarkParseFile-4 245095530 244779103 -0.13%

s/NewNode/newAstNode

b12279c

reduce unused Regexp compile

64a3a04

update benchmarks result in README

3a2f2be

haya14busa added 2 commits September 20, 2016 03:51

return nil for Node(-1)

2ff688a

$ benchcmp prof.3 prof.4 benchmark old ns/op new ns/op delta BenchmarkParseFile-4 227103681 221732445 -2.37%

haya14busa merged commit 30ddaee into master Sep 19, 2016

haya14busa deleted the performance-export branch September 19, 2016 19:22

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Improve performance2: reduce unneeded allocation #16

Improve performance2: reduce unneeded allocation #16

Uh oh!

haya14busa commented Sep 19, 2016 •

edited

Loading

Uh oh!

coveralls commented Sep 19, 2016 •

edited

Loading

Uh oh!

haya14busa commented Sep 19, 2016

Uh oh!

coveralls commented Sep 19, 2016 •

edited

Loading

Uh oh!

coveralls commented Sep 19, 2016 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Improve performance2: reduce unneeded allocation #16

Improve performance2: reduce unneeded allocation #16

Uh oh!

Conversation

haya14busa commented Sep 19, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coveralls commented Sep 19, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

haya14busa commented Sep 19, 2016

Benchmarks

Uh oh!

coveralls commented Sep 19, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coveralls commented Sep 19, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

haya14busa commented Sep 19, 2016 •

edited

Loading

coveralls commented Sep 19, 2016 •

edited

Loading

coveralls commented Sep 19, 2016 •

edited

Loading

coveralls commented Sep 19, 2016 •

edited

Loading