[fix](catalog) refactor the schema cache for external table #34517
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Thank you for your contribution to Apache Doris. Since 2024-03-18, the Document has been moved to doris-website. |
morningman
changed the title
Contributor
Author
|
run buildall |
Contributor
Author
|
run buildall |
morningman
added a commit
to morningman/doris
that referenced
this pull request
May 8, 2024
Contributor
|
PR approved by anyone and no changes requested. |
Contributor
Author
|
run buildall |
TPC-H: Total hot run time: 41740 ms |
TPC-DS: Total hot run time: 185839 ms |
github-actions
bot
added
the
approved
Contributor
|
PR approved by at least one committer and no changes requested. |
morningman
added a commit
to morningman/doris
that referenced
this pull request
May 9, 2024
…4517) Followup apache#33610 Refactor the logic of schema cache for external table. Before, the ExternalSchemaCache only caches the schema(which is `List<Column>`) of the table. But actually, some other objects are also being refreshed when schema cache refreshing, such as `partition columns`, etc. So This PR mainly changes: 1. Add a new class `SchemaCacheValue` as a parent class for all kinds of cache value in ExternalSchemaCache 2. It contains `List<Column>` basically, and sub class can extend from it to add more cached objects. 3. Move all cached objects into `SchemaCacheValue`, so that they will be refreshed along with schema together. Additional: 1. Remove unused warn log in AcceptListener.java 2. Fix unclear error msg if trino connector's table does not exist.
morningman
added a commit
to morningman/doris
that referenced
this pull request
May 9, 2024
…4517) Followup apache#33610 Refactor the logic of schema cache for external table. Before, the ExternalSchemaCache only caches the schema(which is `List<Column>`) of the table. But actually, some other objects are also being refreshed when schema cache refreshing, such as `partition columns`, etc. So This PR mainly changes: 1. Add a new class `SchemaCacheValue` as a parent class for all kinds of cache value in ExternalSchemaCache 2. It contains `List<Column>` basically, and sub class can extend from it to add more cached objects. 3. Move all cached objects into `SchemaCacheValue`, so that they will be refreshed along with schema together. Additional: 1. Remove unused warn log in AcceptListener.java 2. Fix unclear error msg if trino connector's table does not exist.
morningman
added a commit
to morningman/doris
that referenced
this pull request
May 9, 2024
…4517) Followup apache#33610 Refactor the logic of schema cache for external table. Before, the ExternalSchemaCache only caches the schema(which is `List<Column>`) of the table. But actually, some other objects are also being refreshed when schema cache refreshing, such as `partition columns`, etc. So This PR mainly changes: 1. Add a new class `SchemaCacheValue` as a parent class for all kinds of cache value in ExternalSchemaCache 2. It contains `List<Column>` basically, and sub class can extend from it to add more cached objects. 3. Move all cached objects into `SchemaCacheValue`, so that they will be refreshed along with schema together. Additional: 1. Remove unused warn log in AcceptListener.java 2. Fix unclear error msg if trino connector's table does not exist.
morningman
added a commit
that referenced
this pull request
May 9, 2024
ByteYue
pushed a commit
to ByteYue/doris
that referenced
this pull request
May 15, 2024
…4517) Followup apache#33610 Refactor the logic of schema cache for external table. Before, the ExternalSchemaCache only caches the schema(which is `List<Column>`) of the table. But actually, some other objects are also being refreshed when schema cache refreshing, such as `partition columns`, etc. So This PR mainly changes: 1. Add a new class `SchemaCacheValue` as a parent class for all kinds of cache value in ExternalSchemaCache 2. It contains `List<Column>` basically, and sub class can extend from it to add more cached objects. 3. Move all cached objects into `SchemaCacheValue`, so that they will be refreshed along with schema together. Additional: 1. Remove unused warn log in AcceptListener.java 2. Fix unclear error msg if trino connector's table does not exist.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Proposed changes
Followup #33610
Refactor the logic of schema cache for external table.
Before, the ExternalSchemaCache only caches the schema(which is
List<Column>) of the table.But actually, some other objects are also being refreshed when schema cache refreshing,
such as
partition columns, etc.So This PR mainly changes:
SchemaCacheValueas a parent class for all kinds of cache value in ExternalSchemaCacheList<Column>basically, and sub class can extend from it to add more cached objects.SchemaCacheValue, so that they will be refreshed along with schema together.Additional:
Further comments
If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...