Failure in kll_sketch_float_build_k()

[nikunjbhartia]

Query:

SELECT
  group_key,
  count(*) AS total_count,
  bqutil.datasketches.kll_sketch_float_build_k(x, 250) AS kll_sketch
FROM (
  SELECT
    CONCAT("group_key_", CAST(RAND() * 100 AS INT64)) AS group_key,
    RAND() AS x
  FROM
  UNNEST(GENERATE_ARRAY(1, 1000000))) as sample_data
GROUP BY group_key;

BQ Error:
Query error: ReferenceError: sketch is not defined at bqutil.datasketches.kll_sketch_float_build_k(FLOAT64, INT64) line 44, columns 27-28 at [10:1]

Potential issue:
https://github.com/apache/datasketches-bigquery/blob/main/kll/sqlx/kll_sketch_float_build_k.sqlx#L73C6-L73C38

sketch.merge(state.serialized);
sketch referenced without state

[AlexanderSaydakov]

Right. Again this code path after the transition from deserialize back to aggregate, which does not happen often. It is not exercised in small tests.
I believe there was a plan to eliminate this transition. Do you know how is it going?

[AlexanderSaydakov]

And again I could not reproduce this in my environment. The above query runs fine for me. I tried to increase the number in GENERATE, but it fails with "GENERATE_ARRAY(1, 2000000, 1) produced too many elements"

[AlexanderSaydakov]

this should fix the issue, but I am not sure how to test: #137

[nikunjbhartia]

I am not sure about the plan on eliminating the transition.

GENERATE_ARRAY(1, 2000000, 1) produced too many elements
you can do a cross join and generate as many rows you want
select
x, y
from unnest(generate_array(1,1000000)) as x, unnest(generate_array(1,100)) as y

[AlexanderSaydakov]

released and deployed to BigQuery