Skip to content

fix bug with sqlOuterLimit, use sqlOuterLimit in web console#8919

Merged
gianm merged 7 commits intoapache:masterfrom
clintropolis:web-console-sql-limit
Dec 4, 2019
Merged

fix bug with sqlOuterLimit, use sqlOuterLimit in web console#8919
gianm merged 7 commits intoapache:masterfrom
clintropolis:web-console-sql-limit

Conversation

@clintropolis
Copy link
Copy Markdown
Member

Description

This PR modifies the 'smart limit' strategy in the web-console query view to use the sqlOuterLimit parameter introduced in #8566. It also fixes a bug with sqlOuterLimit if the outer rel is already a LogicalSort which would cause a query which should plan to a topN but without a limit set to be unplannable. The logic now will modify the limit of an outer sort instead of wrapping it in another sort.


This PR has:

  • been self-reviewed.
  • added documentation for new or modified features or behaviors.
  • added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
  • added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
  • added unit tests or modified existing tests to cover new code paths.
  • added integration tests.
  • been tested in a test Druid cluster.

@vogievetsky
Copy link
Copy Markdown
Contributor

👍 on the console changes!


if (!isEmptyContext(queryContext)) {
if (!isEmptyContext(queryContext) || wrapQueryLimit) {
jsonQuery.context = Object.assign(jsonQuery.context || {}, queryContext);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

May be better to do:

jsonQuery.context = Object.assign({}, jsonQuery.context || {}, queryContext);

So there is no chance of changing something silly in pleace

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changed


if (!isEmptyContext(queryContext)) explainPayload.context = queryContext;
if (!isEmptyContext(queryContext) || wrapQueryLimit) {
explainPayload.context = queryContext || {};
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
explainPayload.context = queryContext || {};
explainPayload.context = Object.assign({}, queryContext || {});

Otherwise you are changing the queryContext state variable in place if it is set. Maybe not but still cleaner to avoid even the possibility of that.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changed

new Object[]{"abc", 1L}
);
}
testQuery(
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's best to have one testQuery call per test method. So, I'd create a new method for this test.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

split the extra one already there, the one i added into separate tests, also added a few more

return LogicalSort.create(
outerSort.getInput(),
outerSort.collation,
makeBigIntLiteral(0),
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This won't handle offsets properly, nor will it handle properly the case where the outer limit is higher than the inner limit. Check out SortCollapseRule for some logic that handles both of these cases. You might want to create a helper method and use it in both places.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pushed some repeated code to extract fetch and offset and fetch combining logic from SortCollapseRule to Calcites

@vogievetsky
Copy link
Copy Markdown
Contributor

TS side looks great 👍 thank you for addressing my feedback

@clintropolis clintropolis removed the WIP label Nov 27, 2019
Copy link
Copy Markdown
Contributor

@gianm gianm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks @clintropolis

@gianm gianm merged commit d0a6fe7 into apache:master Dec 4, 2019
@gianm gianm added this to the 0.17.0 milestone Dec 4, 2019
@clintropolis clintropolis deleted the web-console-sql-limit branch December 4, 2019 02:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants