Skip to content

[fix](storage) fix core for string predicate in storage layer #9500

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
yiguolei merged 3 commits into from
May 12, 2022
Merged

[fix](storage) fix core for string predicate in storage layer #9500

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
2d8ec6f
fix
May 11, 2022
8db20f8
delete flag
May 11, 2022
4790740
format code
May 12, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 22 additions & 2 deletions be/src/vec/columns/predicate_column.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -230,17 +230,35 @@ class PredicateColumnType final : public COWHelper<IColumn, PredicateColumnType<
void insert_many_binary_data(char* data_array, uint32_t* len_array,
uint32_t* start_offset_array, size_t num) override {
if constexpr (std::is_same_v<T, StringValue>) {
if (_pool == nullptr) {
_pool.reset(new MemPool("PredicateStringColumn"));
}

size_t total_mem_size = 0;
for (size_t i = 0; i < num; i++) {
total_mem_size += len_array[i];
}

char* destination = (char*)_pool->allocate(total_mem_size);
for (size_t i = 0; i < num; i++) {
uint32_t len = len_array[i];
uint32_t start_offset = start_offset_array[i];
insert_string_value(data_array + start_offset, len);
memcpy(destination, data_array + start_offset, len);
Copy link
Contributor

@jacktengg jacktengg May 11, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure how much performnce will be impacted by allocate and memcpy here.

StringValue sv(destination, len);
data.push_back_without_reserve(sv);
destination += len;
}
}
}

void insert_default() override { data.push_back(T()); }

void clear() override { data.clear(); }
void clear() override {
data.clear();
if (_pool != nullptr) {
_pool->clear();
}
}

size_t byte_size() const override { return data.size() * sizeof(T); }

Expand Down Expand Up @@ -410,6 +428,8 @@ class PredicateColumnType final : public COWHelper<IColumn, PredicateColumnType<

private:
Container data;
// manages the memory for slice's data(For string type)
std::unique_ptr<MemPool> _pool;
};
using ColumnStringValue = PredicateColumnType<StringValue>;

Expand Down