Skip to content

Window Functions : Improve performance by comparing Strings in frame bytes without converting them#17091

Merged
kgyrtkirk merged 6 commits intoapache:masterfrom
sreemanamala:win-perf
Sep 19, 2024
Merged

Window Functions : Improve performance by comparing Strings in frame bytes without converting them#17091
kgyrtkirk merged 6 commits intoapache:masterfrom
sreemanamala:win-perf

Conversation

@sreemanamala
Copy link
Copy Markdown
Contributor

Description

After this PR RACs are serialised and transferred over wire.
This commit tries to compare the bytes directly in the NaiveSorter without converting them to Strings.
In combination, these are aiming to reduce the window query times
Also, added a new benchmark with group by __time

previous benchmark results when committed with the Benchmark PR:
image

current benchmark results:
image


Key changed/added classes in this PR
  • StringFrameColumnReader
  • SqlWindowFunctionsBenchmark

This PR has:

  • been self-reviewed.
  • added documentation for new or modified features or behaviors.
  • a release note entry in the PR description.
  • added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
  • added or updated version, license, or notice information in licenses.yaml
  • added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
  • added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
  • added integration tests.
  • been tested in a test Druid cluster.

Copy link
Copy Markdown
Member

@kgyrtkirk kgyrtkirk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

latest change compares null-s correctly

@kgyrtkirk kgyrtkirk merged commit b9a4c73 into apache:master Sep 19, 2024
@sreemanamala sreemanamala deleted the win-perf branch September 19, 2024 08:11
@kgyrtkirk kgyrtkirk added this to the 31.0.0 milestone Oct 9, 2024
kgyrtkirk pushed a commit to kgyrtkirk/druid that referenced this pull request Oct 9, 2024
…s in frame bytes without converting them (apache#17091)

(cherry picked from commit b9a4c73)
kfaraz pushed a commit that referenced this pull request Oct 10, 2024
…s in frame bytes without converting them (#17091) (#17311)

(cherry picked from commit b9a4c73)

Co-authored-by: Sree Charan Manamala <sree.manamala@imply.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants