Skip to content

GroupBy V2 query fails when using a large limit #4739

@jon-wei

Description

@jon-wei

0.10.1 added limit push down optimization to GroupBy V2 in PR #3873.

When the optimization is on, GroupBy V2 requires a buffer with a minimum size depending on the limit.

If the user submits a query with a limit that's large enough such that the configured processing buffers are too small to support this optimization, the query fails with an error like:

 { "errorClass": "io.druid.java.util.common.IAE", "host": "localhost:8083", "errorMessage": "Buffer capacity [34636833] is too small for limit[500000000] with load factor[0.700000], minimum bytes needed: [1666887684]", "error": "Unknown exception" }

It would be better to disable the push down optimization in that situation instead of failing the query.

Metadata

Metadata

Assignees

Labels

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions