Skip to content

Conversation

@Lchangliang
Copy link
Contributor

Proposed changes

Issue Number: close #xxx

core stack
F0821 09:02:39.542575 2297202 segment_loader.h:136] Check failed: segments.empty() 16
*** Check failure stack trace: ***
    @     0x55df1e91b50d  google::LogMessage::Fail()
    @     0x55df1e91da49  google::LogMessage::SendToLog()
    @     0x55df1e91b076  google::LogMessage::Flush()
    @     0x55df1e91e0b9  google::LogMessageFatal::~LogMessageFatal()
    @     0x55defb649fed  doris::SegmentCacheHandle::~SegmentCacheHandle()
    @     0x55defb64e8b4  doris::BetaRowsetReader::~BetaRowsetReader()
    @     0x55defb64e929  doris::BetaRowsetReader::~BetaRowsetReader()
    @     0x55defb617caa  std::_Sp_counted_ptr<>::_M_dispose()
    @     0x55def9ae18d2  std::_Sp_counted_base<>::_M_release()
    @     0x55def9ae160a  std::__shared_count<>::~__shared_count()
    @     0x55def9f81b7b  std::__shared_ptr<>::~__shared_ptr()
    @     0x55def9f76845  std::shared_ptr<>::~shared_ptr()
    @     0x55def9f8c4b7  std::destroy_at<>()
    @     0x55def9f8c487  std::_Destroy<>()
    @     0x55def9f8c43b  std::_Destroy_aux<>::__destroy<>()
    @     0x55def9f8c3df  std::_Destroy<>()
    @     0x55def9f8c2a3  std::_Destroy<>()
    @     0x55def9f707c1  std::vector<>::~vector()
    @     0x55def9f529c8  doris::Compaction::~Compaction()
    @     0x55defc047a87  doris::CumulativeCompaction::~CumulativeCompaction()
    @     0x55defc047aa9  doris::CumulativeCompaction::~CumulativeCompaction()
    @     0x55defbe5ad6a  std::_Sp_counted_ptr<>::_M_dispose()
    @     0x55def9ae18d2  std::_Sp_counted_base<>::_M_release()
    @     0x55def9ae160a  std::__shared_count<>::~__shared_count()
    @     0x55defbe5a9cb  std::__shared_ptr<>::~__shared_ptr()
    @     0x55defc1508fb  std::__shared_ptr<>::reset()
    @     0x55defc0d9695  doris::Tablet::reset_compaction()
    @     0x55def9d3b16d  doris::StorageEngine::_submit_compaction_task()::$_0::operator()()
    @     0x55def9d3af37  std::__invoke_impl<>()
    @     0x55def9d3aea9  _ZSt10__invoke_rIvRZN5doris13StorageEngine23_submit_compaction_taskESt10shared_ptrINS0_6TabletEENS0_14CompactionTypeEbE3$_0JEENSt9enable_ifIX16is_invocable_r_vIT_T0_DpT1_EES9_E4typeEOSA_DpOSB_
    @     0x55def9d3ac2f  std::_Function_handler<>::_M_invoke()
    @     0x55def9c9efe7  std::function<>::operator()()

When do vertical compaction,the _segment_cache_handle in beta_rowset_reader will be inited many time so that the check will failed.

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@Lchangliang
Copy link
Contributor Author

run buildall

@github-actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@hello-stephen
Copy link
Contributor

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.76 seconds
stream load tsv: 552 seconds loaded 74807831229 Bytes, about 129 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select: 29.5 seconds inserted 10000000 Rows, about 338K ops/s
storage size: 17162116707 Bytes

@Lchangliang Lchangliang changed the title [bugfix](vertical-compaction) Only init the _segment_cache_handle in beta_rowset_reader once [bugfix](vertical-compaction) Only can init the SegmentCacheHandle once Aug 21, 2023
@Lchangliang
Copy link
Contributor Author

run buildall

@github-actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@Lchangliang
Copy link
Contributor Author

run buildall

@github-actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@hello-stephen
Copy link
Contributor

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.2 seconds
stream load tsv: 555 seconds loaded 74807831229 Bytes, about 128 MB/s
stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 29.6 seconds inserted 10000000 Rows, about 337K ops/s
storage size: 17161955984 Bytes

@hello-stephen
Copy link
Contributor

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.39 seconds
stream load tsv: 538 seconds loaded 74807831229 Bytes, about 132 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 30 seconds loaded 861443392 Bytes, about 27 MB/s
insert into select: 29.4 seconds inserted 10000000 Rows, about 340K ops/s
storage size: 17162079100 Bytes

@Lchangliang
Copy link
Contributor Author

run buildall

@github-actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@hello-stephen
Copy link
Contributor

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.04 seconds
stream load tsv: 539 seconds loaded 74807831229 Bytes, about 132 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 68 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 29.4 seconds inserted 10000000 Rows, about 340K ops/s
storage size: 17162208081 Bytes

Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 21, 2023
@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

Copy link
Contributor

@xiaokang xiaokang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@xiaokang xiaokang merged commit 8503d26 into apache:master Aug 21, 2023
@xiaokang xiaokang mentioned this pull request Aug 30, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/2.0.1-merged p0_c reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants