Skip to content

fix merging of groupBy subtotal spec results#8109

Merged
gianm merged 20 commits intoapache:masterfrom
himanshug:bugfix
Aug 6, 2019
Merged

fix merging of groupBy subtotal spec results#8109
gianm merged 20 commits intoapache:masterfrom
himanshug:bugfix

Conversation

@himanshug
Copy link
Copy Markdown
Contributor

@himanshug himanshug commented Jul 18, 2019

Fixes #8091

Issue for Future Optimization #8108


This PR has:

  • [ x] been self-reviewed.
  • [ x] added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
  • [x ] added unit tests or modified existing tests to cover new code paths.

@himanshug himanshug added this to the 0.16.0 milestone Jul 18, 2019
Copy link
Copy Markdown
Contributor

@gianm gianm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks correct to me, my only comments are around making the code clearer & match up with the nice comment you wrote. I had some suggestions but I'm open to other ideas.


return Sequences.withBaggage(result, () -> Lists.reverse(closeables).forEach(closeable -> CloseQuietly.close(closeable)));
}
}, subtotalQuery, null),
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something looks weird about this formatting…

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it was passing the intellij formatter , introduced few newlines and it looks better now.

return GroupByRowProcessor.getRowsFromGrouper(
queryWithoutSubtotalsSpec,
subtotalSpec,
grouperSupplier
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is Grouper#1 right? Could you please add a comment where it's initialized, or change its name (grouperOneSupplier or something) to make the logic easier to follow?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or maybe breaking the method up would make it easier.

I guess what I'm trying to say is when reading, I was wishing it was more clear when we were using Grouper#1, when Grouper#2, and also who's responsible for closing what.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

renamed variable names to align them better with the comment, it should be easier to follow now.

I thought of breaking the method and then realized most of it is actually method calls which are big because of the number of arguments they take. take a look now, if it still looks difficult then I can try to break it.

Sequence<Row> queryResult
)
{
// How it works?
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for writing this comment, it helped.

Sequence<Row> result = GroupByRowProcessor.getRowsFromGrouper(
subtotalQuery,
subtotalSpec,
Suppliers.memoize(() -> GroupByRowProcessor.createGrouper(
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this is Grouper#2. Could you please make it clear via comment, naming, or splitting out into a separate method?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changed

@himanshug himanshug force-pushed the bugfix branch 3 times, most recently from 968ecf7 to 7c2efc5 Compare July 23, 2019 17:24
@himanshug
Copy link
Copy Markdown
Contributor Author

strict build is broken due to #8138

@himanshug
Copy link
Copy Markdown
Contributor Author

@gianm can you please review it again ?

Copy link
Copy Markdown
Contributor

@gianm gianm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

@gianm gianm merged commit 4507a4f into apache:master Aug 6, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

groupBy with subtotalsSpec doesn't fully group each set

2 participants