Skip to content

feat(java): expose sql api for java api#4328

Merged
yanghua merged 11 commits intolance-format:mainfrom
fangbo:java-sql
Jul 30, 2025
Merged

feat(java): expose sql api for java api#4328
yanghua merged 11 commits intolance-format:mainfrom
fangbo:java-sql

Conversation

@fangbo
Copy link
Copy Markdown
Contributor

@fangbo fangbo commented Jul 29, 2025

Add sql query for java api.

epic: #3950

See:

#4267
#4086

@github-actions github-actions Bot added enhancement New feature or request java labels Jul 29, 2025
@fangbo
Copy link
Copy Markdown
Contributor Author

fangbo commented Jul 29, 2025

@yanghua Please review this pr

}
}

public SqlQuery sql(String sql, String tableName) {
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In rust, we make the tableName an optional arg. If users do not specify, the default name is dataset.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In rust, we make the tableName an optional arg. If users do not specify, the default name is dataset.

If the name of the Dataset is custom and the tableName is not specified, the query will report an error. Therefore, I think it is necessary to enforce the specification of the correct tableName in terms of the interface's semantics to avoid misunderstandings by users.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this would be challenged if rust or python doesn't have a table name parameter.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tableName has been changed to Optional in order to keep consistency with python and rust.

reader.close();

// Test agg query
reader = dataset.sql("select sum(id) from " + NAME, NAME).intoBatchRecords();
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we also verify the result of the sum operation?

Comment thread java/core/src/test/java/com/lancedb/lance/SqlQueryTest.java
@yanghua
Copy link
Copy Markdown
Collaborator

yanghua commented Jul 29, 2025

@fangbo, thanks for your contribution! Left some comments.

private boolean withRowId = false;
private boolean withRowAddr = false;

public SqlQuery(BufferAllocator allocator, Dataset dataset, String sql, String table) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we may reuse allocator in the dataset? The issue here is who could be responsable for releasing this allocator

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The allocator is reused from Dataset's allocator.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean this parameter of allocator may be unnecessary and expose some risk. We could directly get it from dataset.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it mean that Dataset provides a new method like getAllocator() which returns the Allocator, and SqlQuery uses getAllocator() to get the shared Allocator ?

Copy link
Copy Markdown
Contributor

@majin1102 majin1102 Jul 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ye, that should be an option.

I think right now you could just use dataset.allocator if you put that code in the same package. There has been use cases like this.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good suggestion. The SqlQuery has been move to package com.lancedb.lance and it uses dataset.allocator.

}
}

public SqlQuery sql(String sql, String tableName) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this would be challenged if rust or python doesn't have a table name parameter.

Comment thread java/core/src/main/java/com/lancedb/lance/SqlQuery.java Outdated
Copy link
Copy Markdown
Collaborator

@yanghua yanghua left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, if we can add a comment for the public api. It would be better.

Comment thread java/core/src/main/java/com/lancedb/lance/Dataset.java
Copy link
Copy Markdown
Contributor

@jackye1995 jackye1995 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good to me, +1 for adding some comments since this is public API.

@yanghua yanghua merged commit 2d57f22 into lance-format:main Jul 30, 2025
8 checks passed
@fangbo fangbo deleted the java-sql branch July 31, 2025 08:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request java

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants