Skip to content

Comments

Map test cleanup#727

Merged
gdbelvin merged 7 commits intogoogle:masterfrom
gdbelvin:integration
Jul 10, 2017
Merged

Map test cleanup#727
gdbelvin merged 7 commits intogoogle:masterfrom
gdbelvin:integration

Conversation

@gdbelvin
Copy link
Contributor

@gdbelvin gdbelvin commented Jul 7, 2017

In order to facilitate easier debugging and prepare for the needed changes to support #691, I did some preliminary code cleanup:

  • Use canonical sort functions in HStar2
  • Use hasher.BitLen() in place of hasher.Size()*8
  • Use index ([]byte)/ value pairs in tests instead of key (string)/value pairs.
    • This removed a dependency on testonly.HashKey
    • map[key]value -> [][]byte of alternating index / value pairs.
  • Removed unused test vectors
  • Combined EmptyRoot* tests into simpleTestVector


const treeID = int64(0)

var hasher = maphasher.Default
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why move this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I moved the hasher variable to a package variable in order to make testing with a different hasher easier - one place to change it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't we need a test matrix for that though?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To do it properly, yes.

In the next PR, I'll be adding a MapEnv we can can do this with a table driven test.

if err != nil {
t.Fatalf("Failed to calculate root: %v", err)
// createHStar2Leaves returns a []HStar2LeafHash formed by the mapping of index, value ...
// createHStar2Leaves panics if len(iv) is odd. Duplicate i/v pairs get over written.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you reference the doc that this came from and/or explain how we know the results are reliable?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This algorithms is inspired by gRPC's [metadata.Pairs](https://github.com/grpc/grpc-go/blob/master/metadata/metadata.go#L65) which also has a key / value pair format.

I use a map internally to also guarantee that emitted values don't have duplicate keys.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we're at cross purposes. I meant that before we had hardcoded values that couldn't go wrong. If this leaf generation code code is basically a copy of the code we're testing for example then it's not ideal.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh, hmm.

This isn't a new function - it replaces the previous createHStar2Leaves which was also in the test. It's just a test helper function. There's nothing like this function in the non-test codebase.

}
}

// b64 converts a base64 string into []byte.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The name b64 sounds like an encoder rather than a decoder.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would fromB64 be better? I was hoping to keep a short name because this function gets used a lot when forming table driven tests and a long name just pushes the test vectors off the screen.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unB64 or deB64?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

deB64 sounds good.

},
testonly.MustDecodeBase64("U6ANU1en3BSbbnWqhV2nTGtQ+scBlaZf9kRPEEDZsHM="),
}, {
"key-1-848",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we reducing coverage by removing these cases?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're not losing any coverage. The cases weren't being used, and I don't think they were exercising any additional code paths.

In the next PR, I'll be adding a fuller suite of merkle tree tests that don't rely on test vectors, and can thus be used with any hasher.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK.

@Martin2112
Copy link
Contributor

Can you fix Travis and then it's probably ready.

@gdbelvin
Copy link
Contributor Author

Looks like it was the mysql driver being flakey.

@gdbelvin gdbelvin merged commit 9d4b679 into google:master Jul 10, 2017
@gdbelvin gdbelvin deleted the integration branch July 10, 2017 10:42
@gdbelvin gdbelvin mentioned this pull request Aug 9, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants