add hyperloglog implementation (add and count)

[jimexist]

Which issue does this PR close?

Related #1087 and serves as a pre-cursor to that pull request
Closes #.

Rationale for this change

I'm thinking of adopting Redis's hyperloglog implementation along with https://github.com/crepererum/pdatastructs.rs/blob/master/src/hyperloglog.rs for its code structure.

I don't necessarily want to use @crepererum pdatastructs as is because:

1. this way the code size of datafusion is well controlled and since we can make some reasonably good strong assumption on the usage pattern, we can skip some of the generics and adopt Redis's approach (e.g. precision = 14 with 16384 registers, but also each with u8 size instead of the raw byte array in C in order to save space, trading off with rather tricky bit shifting techniques - this way instead of max size of 12KiB we'll have 16KiB per HLL)
2. also we need to have either bincode serde so that merge can work across record batches

What changes are included in this PR?

Are there any user-facing changes?

[Dandandan]

As we're using only aHash in this usage here, we could use the aHash methods and avoid the generic hash builder api.

[houqp]

nice work!

[Dandandan]

I think the builder or randomstate has to be reused across calls, otherwise it will generate different hashes across calls for the same value.

[jimexist]

And wouldn't that be okay and also contribute to probabilistic stability?

[jimexist]

After the second thought that was a good point so let me change this

[crepererum]

You have to use the same hash function for the same session (or for all sessions). If not, then your hash function is more or less a random number generator (in the worst case) and you're (depending on how many hash functions you generate) overestimate cardinality (because the same input now creates different hashes and looks like different inputs).

What you do here is half-correct: AHasher::default uses either a compile-time seed or a process-time seed (once drawn using global state). This behavior is OK as long as the query is local and you don't serialize the HLL. However, if you have multiple processes (e.g. ballista) or someone is using serde to dump and restore the HLL, this breaks because now you have a totally different hash function.

To future-proof this whole thing and to make DoS attacks less likely, I would advice you to make the hasher a runtime parameter of the HLL, generate it once during query planning and serialize the hasher alongside the query plan and the intermediate HLLs.

[Dandandan]

For now maybe a fixed RandomState would be OK-ish? In the join / aggregate algorithms we are also doing this. @houqp do you have an opinion on this? I guess usages like ROAPI have a higher chance of being vulnerable to DoS attacks by exposing an API to the end user. We should do something like what @crepererum suggests to make it work for e.g. ballista.
There are of course more parts of the query engine you could abuse for DoS attacks, like generating cross joins, having a very large number of columns, etc. so maybe it makes more sense to spend some time in being able to e.g. specify and handle timeouts for query execution.

[jimexist]

All are good points thanks for the discussion. I've decided to use a static random state for the moment and revisit the decision once cross session serialization is in scope.

[alamb]

This is some cool work @jimexist

[chitralverma]

@jimexist how do you think the current implementation compares with this crate?