Skip to content

check lengths of maps and recurse over only one if it is necessary#87

Merged
evanphx merged 2 commits intoevanphx:masterfrom
logicalhan:optimize
Aug 15, 2019
Merged

check lengths of maps and recurse over only one if it is necessary#87
evanphx merged 2 commits intoevanphx:masterfrom
logicalhan:optimize

Conversation

@logicalhan
Copy link
Copy Markdown

@logicalhan logicalhan commented Aug 15, 2019

The matchesValue function, in the worst case, has exponential time complexity (for deeply nested maps). This fix should make things operate in time linear to the number of objects compared.

Tests added: I've added some benchmark tests to justify the change. The tests generate a number of nested maps which creates an exponential number of nested map objects, given some input n. More precisely, the helper function generates 2^(n+1)-1 objects.

We can see that with the original function, the growth does not increase linearly with the number of objects:

go test -bench=.                                                                                                                 1 ↵
goos: linux
goarch: amd64
pkg: github.com/evanphx/json-patch
BenchmarkMatchesValue1/objectCount=3-6         	10000000	       180 ns/op
BenchmarkMatchesValue2/objectCount=7-6         	 2000000	       881 ns/op
BenchmarkMatchesValue3/objectCount=15-6        	  300000	      3742 ns/op
BenchmarkMatchesValue4/objectCount=31-6        	  100000	     15110 ns/op
BenchmarkMatchesValue5/objectCount=63-6        	   20000	     59887 ns/op
BenchmarkMatchesValue6/objectCount=127-6       	    5000	    243844 ns/op
BenchmarkMatchesValue7/objectCount=255-6       	    2000	    968795 ns/op
BenchmarkMatchesValue8/objectCount=511-6       	     300	   3944336 ns/op
BenchmarkMatchesValue9/objectCount=1023-6      	     100	  15553770 ns/op
BenchmarkMatchesValue10/objectCount=2047-6     	      20	  62673221 ns/op

Which makes sense, since the matchesValue function is currently exponential for nested maps, since we recursively call matchesValue twice for each key. With the patch, the operation scales linearly to in the increased object count.

╰─$ go test -bench=.                                                                                                               130 ↵
goos: linux
goarch: amd64
pkg: github.com/evanphx/json-patch
BenchmarkMatchesValue1/objectCount=3-6         	20000000	        83.0 ns/op
BenchmarkMatchesValue2/objectCount=7-6         	 5000000	       250 ns/op
BenchmarkMatchesValue3/objectCount=15-6        	 3000000	       592 ns/op
BenchmarkMatchesValue4/objectCount=31-6        	 1000000	      1274 ns/op
BenchmarkMatchesValue5/objectCount=63-6        	  500000	      2641 ns/op
BenchmarkMatchesValue6/objectCount=127-6       	  200000	      5414 ns/op
BenchmarkMatchesValue7/objectCount=255-6       	  100000	     11117 ns/op
BenchmarkMatchesValue8/objectCount=511-6       	  100000	     22247 ns/op
BenchmarkMatchesValue9/objectCount=1023-6      	   30000	     44813 ns/op
BenchmarkMatchesValue10/objectCount=2047-6     	   20000	     89374 ns/op

@logicalhan
Copy link
Copy Markdown
Author

/cc @evanphx

Whenever you have the time to look this over, it would be much appreciated.

@evanphx evanphx merged commit e83c0a1 into evanphx:master Aug 15, 2019
@evanphx
Copy link
Copy Markdown
Owner

evanphx commented Aug 15, 2019

Thanks so much! Great catch!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants