Refactor fulltext search #2358
Merged
Refactor fulltext search #2358
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…xera/texera into shengquan-fix-fulltext-search
Yicong-Huang
requested changes
Feb 16, 2024
Contributor
Yicong-Huang
left a comment
Yicong-Huang
left a comment
There was a problem hiding this comment.
The refactored feature looks pretty good! I think there are some comments to address.
.../amber/src/main/scala/edu/uci/ics/texera/web/resource/dashboard/FileSearchQueryBuilder.scala
Outdated
Show resolved
Hide resolved
...mber/src/main/scala/edu/uci/ics/texera/web/resource/dashboard/FulltextSearchQueryUtils.scala
Outdated
Show resolved
Hide resolved
...mber/src/main/scala/edu/uci/ics/texera/web/resource/dashboard/FulltextSearchQueryUtils.scala
Outdated
Show resolved
Hide resolved
core/amber/src/main/scala/edu/uci/ics/texera/web/resource/dashboard/SearchQueryBuilder.scala
Outdated
Show resolved
Hide resolved
core/amber/src/main/scala/edu/uci/ics/texera/web/resource/dashboard/UnifiedResourceSchema.scala
Outdated
Show resolved
Hide resolved
...mber/src/main/scala/edu/uci/ics/texera/web/resource/dashboard/FulltextSearchQueryUtils.scala
Show resolved
Hide resolved
Yicong-Huang
approved these changes
Feb 23, 2024
core/amber/src/main/scala/edu/uci/ics/texera/web/resource/dashboard/DashboardResource.scala
Show resolved
Hide resolved
...mber/src/main/scala/edu/uci/ics/texera/web/resource/dashboard/FulltextSearchQueryUtils.scala
Outdated
Show resolved
Hide resolved
core/amber/src/main/scala/edu/uci/ics/texera/web/resource/dashboard/SearchQueryBuilder.scala
Outdated
Show resolved
Hide resolved
This was referenced Mar 21, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
IMPORTANT NOTE: Due to the introduction of dataset in #2391, we need to add a new dataset search query builder to this PR. However, the dataset changes are not merged completely yet, we decide not to support dataset as a searchable resource. After the dataset changes are all merged, we need another PR to add the support.
Background:
In Texera, we have 3 resource types: File, Workflow and Project. Each resource has its access control and schema. We want to be able to search all resources using a single search input box. Our approach is to create a unified schema by union all different schemas to create a big query. This approach also simplifies the handling of
offsetandlimit.Refactoring:
This PR breaks down the construction of full-text search query into several components for better maintainability:
FulltextSearchQueryUtilscontains helper functions to formulatewhereconditions given query parameters.UnifiedResourceSchemaprovides the unified schema, each resource maps its own schema to the unified schema.SearchQueryBuilderprovides a general framework to build a search query of one type of resource. We haveFileSearchQueryBuilder,ProjectSearchQueryBuilderandWorkflowSearchQueryBuilderfor the existing 3 resource types.searchAllResourcesinDashBoardResouceis the endpoint of the full-text search. It unifies all the results and returns them to the front end.Note:
To improve the quality of search results. I added a substring search(
LIKE) condition to the query. If the performance is downgraded by this, we should remove it.