Skip to content

Refactor: Move some classes from sql to processing & server for reusability#17542

Merged
abhishekrb19 merged 12 commits intoapache:masterfrom
abhishekrb19:refactor_broker_classes
Dec 6, 2024
Merged

Refactor: Move some classes from sql to processing & server for reusability#17542
abhishekrb19 merged 12 commits intoapache:masterfrom
abhishekrb19:refactor_broker_classes

Conversation

@abhishekrb19
Copy link
Copy Markdown
Contributor

This PR contains non-functional / refactoring changes of the following classes in the sql module:

  1. Move ExplainPlan and ExplainAttributes fromsql/src/main/java/org/apache/druid/sql/http to processing/src/main/java/org/apache/druid/query/explain
  2. Move sql/src/main/java/org/apache/druid/sql/SqlTaskStatus.java -> processing/src/main/java/org/apache/druid/query/http/SqlTaskStatus.java
  3. Add a new class processing/src/main/java/org/apache/druid/query/http/ClientSqlQuery.java that is effectively a thin POJO version of SqlQuery in the sql module but without any of the Calcite functionality and business logic.
  4. Move BrokerClient, BrokerClientImpl and Broker classes from sql/src/main/java/org/apache/druid/sql/client to server/src/main/java/org/apache/druid/client/broker.
  5. Remove BrokerServiceModule that provided the BrokerClient. The functionality is now contained in ServiceClientModule in the server package itself which provides all the clients as well.

This is done so that we can reuse the said classes in #17353 without brining in Calcite and other dependencies to the Overlord.

This PR has:

  • been self-reviewed.
  • added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
  • added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
  • been tested in a test Druid cluster.

@github-actions github-actions Bot added Area - Batch Ingestion Area - Querying Area - MSQ For multi stage queries - https://github.com/apache/druid/issues/12262 labels Dec 6, 2024
@abhishekrb19 abhishekrb19 changed the title Refactor: Move some classes in sql to processing & server for reusability Refactor: Move some classes from sql to processing & server for reusability Dec 6, 2024
Copy link
Copy Markdown
Contributor

@kfaraz kfaraz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, minor suggestion for tests.

Comment thread sql/src/main/java/org/apache/druid/sql/http/SqlQuery.java
public class ClientSqlQueryTest
{
@Test
public void testSerde() throws JsonProcessingException
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There should be a test in SqlQueryTest to test inter-conversion between SqlQuery and ClientSqlQuery.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Also, fixed a related bug with parameters caught in the unit test.

@abhishekrb19 abhishekrb19 force-pushed the refactor_broker_classes branch from ea7a58d to 0616081 Compare December 6, 2024 06:18
Copy link
Copy Markdown
Contributor

@kfaraz kfaraz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@abhishekrb19
Copy link
Copy Markdown
Contributor Author

Thanks for the review, @kfaraz. Merging the PR as the failing ITs are unrelated (#17543 should address the flakiness to an extent).

@abhishekrb19 abhishekrb19 merged commit 3a2220c into apache:master Dec 6, 2024
@abhishekrb19 abhishekrb19 deleted the refactor_broker_classes branch December 6, 2024 17:32
@adarshsanjeev adarshsanjeev added this to the 32.0.0 milestone Jan 16, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Area - Batch Ingestion Area - MSQ For multi stage queries - https://github.com/apache/druid/issues/12262 Area - Querying

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants