Skip to content

[BUG] ReAgg when adding agg mv on dup base table #4587

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
morningman merged 2 commits into from
Sep 13, 2020
Merged

[BUG] ReAgg when adding agg mv on dup base table #4587

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
4d872ae
[BUG] Reagg when adding agg mv on dup base table
EmmyMiao87 Sep 10, 2020
f7a1106
Add another case
EmmyMiao87 Sep 11, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 25 additions & 2 deletions be/src/olap/schema_change.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2090,14 +2090,37 @@ OLAPStatus SchemaChangeHandler::_parse_request(TabletSharedPtr base_tablet,
}
}

const TabletSchema& ref_tablet_schema = base_tablet->tablet_schema();
const TabletSchema& new_tablet_schema = new_tablet->tablet_schema();
if (ref_tablet_schema.keys_type() != new_tablet_schema.keys_type()) {
// only when base table is dup and mv is agg
// the rollup job must be reagg.
*sc_sorting = true;
return OLAP_SUCCESS;
}

// If the sort of key has not been changed but the new keys num is less then base's,
// the new table should be re agg.
// So we also need to set sc_sorting = true.
// A, B, C are keys(sort keys), D is value
// followings need resort:
// old keys: A B C D
// new keys: A B
if (new_tablet_schema.keys_type() != KeysType::DUP_KEYS
&& new_tablet->num_key_columns() < base_tablet->num_key_columns()) {
// this is a table with aggregate key type, and num of key columns in new schema
// is less, which means the data in new tablet should be more aggregated.
// so we use sorting schema change to sort and merge the data.
*sc_sorting = true;
return OLAP_SUCCESS;
}

if (base_tablet->num_short_key_columns() != new_tablet->num_short_key_columns()) {
// the number of short_keys changed, can't do linked schema change
*sc_directly = true;
return OLAP_SUCCESS;
}

const TabletSchema& ref_tablet_schema = base_tablet->tablet_schema();
const TabletSchema& new_tablet_schema = new_tablet->tablet_schema();
for (size_t i = 0; i < new_tablet->num_columns(); ++i) {
ColumnMapping* column_mapping = rb_changer->get_mutable_column_mapping(i);
if (column_mapping->ref_column < 0) {
Expand Down