Skip to content

fix nested column thread safety issue for 24.0.1#13267

Merged
abhishekagarwal87 merged 1 commit intoapache:24.0.1from
clintropolis:nested-column-thread-safety
Oct 28, 2022
Merged

fix nested column thread safety issue for 24.0.1#13267
abhishekagarwal87 merged 1 commit intoapache:24.0.1from
clintropolis:nested-column-thread-safety

Conversation

@clintropolis
Copy link
Copy Markdown
Member

smaller footprint version of fix #13265 for 24.0.1, this just fixes the TypeStrategy to use positional reads

@clintropolis clintropolis changed the title fix nested column thread safety issue fix nested column thread safety issue for 24.0.1 Oct 27, 2022
Comment on lines +165 to +166
indexer.mergeFields(mergedFields);
return indexer.getSortedCollector();
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change is just general refactoring and not actually functionally meaningful, right? Just validating my understanding

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep, made writing the test easier and the direct field access was kind of ugly

@abhishekagarwal87 abhishekagarwal87 merged commit 542277e into apache:24.0.1 Oct 28, 2022
@xvrl
Copy link
Copy Markdown
Member

xvrl commented Oct 28, 2022

@clintropolis it looks like integration tests are still failing in the 24.0.1 branch. We might want to check if there is something non-deterministic or not.

@clintropolis
Copy link
Copy Markdown
Member Author

it looks like integration tests are still failing in the 24.0.1 branch. We might want to check if there is something non-deterministic or not.

hmm, these actually look unrelated to my fix, but maybe worth looking into

[ERROR] org.apache.druid.sql.avatica.DruidAvaticaHandlerTest.testConcurrentQueries  Time elapsed: 12.981 s  <<< ERROR!
java.util.concurrent.ExecutionException: java.lang.RuntimeException: org.apache.calcite.avatica.AvaticaSqlException: Error -1 (00000) : Error while executing SQL "SELECT COUNT(*) + 1636 AS ci FROM foo": Remote driver error: QueryInterruptedException: Error while applying rule DruidQueryRule(AGGREGATE_PROJECT), args [rel#631993:LogicalProject.NONE.[](input=RelSubset#631992,ci=+($0, 1636)), rel#632012:DruidQueryRel.NONE.[](query={"queryType":"timeseries","dataSource":{"type":"table","name":"foo"},"intervals":{"type":"intervals","intervals":["-146136543-09-08T08:23:32.096Z/146140482-04-24T15:36:27.903Z"]},"granularity":{"type":"all"},"aggregations":[{"type":"count","name":"a0"}],"context":{"nativeQueryIds":"[3ef1e7e1-f2f6-4152-8384-229eaea39482]","sqlQueryId":"f2727102-ff6c-404c-8f9a-2ed84a406e46","sqlStringifyArrays":false}},signature={a0:LONG})] -> RuntimeException: Error while applying rule DruidQueryRule(AGGREGATE_PROJECT), args [rel#631993:LogicalProject.NONE.[](input=RelSubset#631992,ci=+($0, 1636)), rel#632012:DruidQueryRel.NONE.[](query={"queryType":"timeseries","dataSource":{"type":"table","name":"foo"},"intervals":{"type":"intervals","intervals":["-146136543-09-08T08:23:32.096Z/146140482-04-24T15:36:27.903Z"]},"granularity":{"type":"all"},"aggregations":[{"type":"count","name":"a0"}],"context":{"nativeQueryIds":"[3ef1e7e1-f2f6-4152-8384-229eaea39482]","sqlQueryId":"f2727102-ff6c-404c-8f9a-2ed84a406e46","sqlStringifyArrays":false}},signature={a0:LONG})] -> NullPointerException: Cannot invoke "java.util.Map.size()" because "m" is null

I think maybe #13049 is the fix

@clintropolis clintropolis deleted the nested-column-thread-safety branch October 28, 2022 22:45
@clintropolis
Copy link
Copy Markdown
Member Author

created #13281 to backport that fix

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants