test(bindings/c): Implement valgrind check in test and ci#2653
Merged
Xuanwo merged 4 commits intoapache:mainfrom Jul 20, 2023
Merged
test(bindings/c): Implement valgrind check in test and ci#2653Xuanwo merged 4 commits intoapache:mainfrom
Xuanwo merged 4 commits intoapache:mainfrom
Conversation
wildbartty
requested review from
ClSlaid,
PsiACE,
Xuanwo and
sundy-li
as code owners
Member
|
Bravo! It seems that valgrind helps us catch a memory leak. cc @Ji-Xinyou, would you like to take a look and have a fix? ==6590== 16 bytes in 1 blocks are definitely lost in loss record 1 of 1
==6590== at 0x4848899: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==6590== by 0x49A653E: alloc::alloc::Global::alloc_impl (alloc.rs:102)
==6590== by 0x49A63F3: alloc::alloc::exchange_malloc (alloc.rs:245)
==6590== by 0x49A52F5: opendal_operator_blocking_read (boxed.rs:217)
==6590== by 0x1159A3: OpendalBddTest_FeatureTest_Test::TestBody() (in /home/runner/work/incubator-opendal/incubator-opendal/bindings/c/build/bdd)
==6590== by 0x149DBE: void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) (in /home/runner/work/incubator-opendal/incubator-opendal/bindings/c/build/bdd)
==6590== by 0x13DB55: testing::Test::Run() (in /home/runner/work/incubator-opendal/incubator-opendal/bindings/c/build/bdd)
==6590== by 0x13DCD4: testing::TestInfo::Run() (in /home/runner/work/incubator-opendal/incubator-opendal/bindings/c/build/bdd)
==6590== by 0x13E288: testing::TestSuite::Run() (in /home/runner/work/incubator-opendal/incubator-opendal/bindings/c/build/bdd)
==6590== by 0x13E98E: testing::internal::UnitTestImpl::RunAllTests() (in /home/runner/work/incubator-opendal/incubator-opendal/bindings/c/build/bdd)
==6590== by 0x14A[38](https://github.com/apache/incubator-opendal/actions/runs/5575185496/jobs/10184759670?pr=2653#step:7:39)6: bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) (in /home/runner/work/incubator-opendal/incubator-opendal/bindings/c/build/bdd)
==6590== by 0x13DD9B: testing::UnitTest::Run() (in /home/runner/work/incubator-opendal/incubator-opendal/bindings/c/build/bdd) |
Member
|
This PR mostly LGTM! We can merge this PR after the memory leak fixed (in other PR). |
Contributor
Definitely! I would take a look as soon as I get my hands on my machine. Thanks for the work🥰 |
Member
|
There are also leaks in list test: valgrind --error-exitcode=1 --leak-check=full -- ./build/list
==309525== Memcheck, a memory error detector
==309525== Copyright (C) 2002-2022, and GNU GPL'd, by Julian Seward et al.
==309525== Using Valgrind-3.21.0 and LibVEX; rerun with -h for copyright info
==309525== Command: ./build/list
==309525==
[==========] Running 3 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 3 tests from OpendalListTest
[ RUN ] OpendalListTest.ListDirTest
==309525== Invalid read of size 1
==309525== at 0x4847ED4: strlen (vg_replace_strmem.c:501)
==309525== by 0x4E30412: core::ffi::c_str::CStr::from_ptr (c_str.rs:293)
==309525== by 0x4E3465A: opendal_operator_stat (lib.rs:416)
==309525== by 0x10BA63: OpendalListTest_ListDirTest_Test::TestBody() (in /home/xuanwo/Code/apache/incubator-opendal/bindings/c/build/list)
==309525== by 0x5FDCB3B: UnknownInlinedFun (gtest.cc:2621)
==309525== by 0x5FDCB3B: void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) [clone .constprop.0] (gtest.cc:2657)
==309525== by 0x5FC9197: UnknownInlinedFun (gtest.cc:2696)
==309525== by 0x5FC9197: testing::Test::Run() (gtest.cc:2686)
==309525== by 0x5FC93A6: testing::TestInfo::Run() (gtest.cc:2845)
==309525== by 0x5FC9503: UnknownInlinedFun (gtest.cc:3004)
==309525== by 0x5FC9503: testing::TestSuite::Run() (gtest.cc:2977)
==309525== by 0x5FD5A6E: testing::internal::UnitTestImpl::RunAllTests() (gtest.cc:5890)
==309525== by 0x5FD46A8: UnknownInlinedFun (gtest.cc:2621)
==309525== by 0x5FD46A8: UnknownInlinedFun (gtest.cc:2657)
==309525== by 0x5FD46A8: testing::UnitTest::Run() (gtest.cc:5455)
==309525== by 0x10CAC3: RUN_ALL_TESTS() (in /home/xuanwo/Code/apache/incubator-opendal/bindings/c/build/list)
==309525== by 0x10C2F5: main (in /home/xuanwo/Code/apache/incubator-opendal/bindings/c/build/list)
==309525== Address 0x659a751 is 0 bytes after a block of size 65 alloc'd
==309525== at 0x4841848: malloc (vg_replace_malloc.c:431)
==309525== by 0x575FF6D: alloc (alloc.rs:102)
==309525== by 0x575FF6D: alloc::alloc::Global::alloc_impl (alloc.rs:185)
==309525== by 0x5760998: <alloc::alloc::Global as core::alloc::Allocator>::allocate (alloc.rs:245)
==309525== by 0x575C09B: alloc::raw_vec::RawVec<T,A>::allocate_in (raw_vec.rs:184)
==309525== by 0x575E9EB: with_capacity_in<u8, alloc::alloc::Global> (raw_vec.rs:130)
==309525== by 0x575E9EB: with_capacity_in<u8, alloc::alloc::Global> (mod.rs:672)
==309525== by 0x575E9EB: <T as alloc::slice::hack::ConvertVec>::to_vec (slice.rs:162)
==309525== by 0x4E9F84B: to_vec<u8, alloc::alloc::Global> (slice.rs:111)
==309525== by 0x4E9F84B: to_vec_in<u8, alloc::alloc::Global> (slice.rs:441)
==309525== by 0x4E9F84B: to_vec<u8> (slice.rs:416)
==309525== by 0x4E9F84B: to_owned<u8> (slice.rs:823)
==309525== by 0x4E9F84B: to_owned (str.rs:209)
==309525== by 0x4E9F84B: from (string.rs:2667)
==309525== by 0x4E9F84B: <str as alloc::string::ToString>::to_string (string.rs:2604)
==309525== by 0x53A7CBF: opendal::raw::oio::entry::Entry::new (entry.rs:37)
==309525== by 0x54BAE78: opendal::raw::adapters::typed_kv::backend::KvPager::inner_next_page::{{closure}} (backend.rs:339)
==309525== by 0x53B1DD1: core::iter::adapters::map::map_fold::{{closure}} (map.rs:84)
==309525== by 0x52DBC4A: core::iter::traits::iterator::Iterator::fold (iterator.rs:2481)
==309525== by 0x53AAC23: <core::iter::adapters::map::Map<I,F> as core::iter::traits::iterator::Iterator>::fold (map.rs:124)
==309525== by 0x53AD95E: core::iter::traits::iterator::Iterator::for_each (iterator.rs:856)
==309525==
==309525== Invalid read of size 1
==309525== at 0x4849018: strcmp (vg_replace_strmem.c:939)
==309525== by 0x10BB7C: OpendalListTest_ListDirTest_Test::TestBody() (in /home/xuanwo/Code/apache/incubator-opendal/bindings/c/build/list)
==309525== by 0x5FDCB3B: UnknownInlinedFun (gtest.cc:2621)
==309525== by 0x5FDCB3B: void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) [clone .constprop.0] (gtest.cc:2657)
==309525== by 0x5FC9197: UnknownInlinedFun (gtest.cc:2696)
==309525== by 0x5FC9197: testing::Test::Run() (gtest.cc:2686)
==309525== by 0x5FC93A6: testing::TestInfo::Run() (gtest.cc:2845)
==309525== by 0x5FC9503: UnknownInlinedFun (gtest.cc:3004)
==309525== by 0x5FC9503: testing::TestSuite::Run() (gtest.cc:2977)
==309525== by 0x5FD5A6E: testing::internal::UnitTestImpl::RunAllTests() (gtest.cc:5890)
==309525== by 0x5FD46A8: UnknownInlinedFun (gtest.cc:2621)
==309525== by 0x5FD46A8: UnknownInlinedFun (gtest.cc:2657)
==309525== by 0x5FD46A8: testing::UnitTest::Run() (gtest.cc:5455)
==309525== by 0x10CAC3: RUN_ALL_TESTS() (in /home/xuanwo/Code/apache/incubator-opendal/bindings/c/build/list)
==309525== by 0x10C2F5: main (in /home/xuanwo/Code/apache/incubator-opendal/bindings/c/build/list)
==309525== Address 0x659a751 is 0 bytes after a block of size 65 alloc'd
==309525== at 0x4841848: malloc (vg_replace_malloc.c:431)
==309525== by 0x575FF6D: alloc (alloc.rs:102)
==309525== by 0x575FF6D: alloc::alloc::Global::alloc_impl (alloc.rs:185)
==309525== by 0x5760998: <alloc::alloc::Global as core::alloc::Allocator>::allocate (alloc.rs:245)
==309525== by 0x575C09B: alloc::raw_vec::RawVec<T,A>::allocate_in (raw_vec.rs:184)
==309525== by 0x575E9EB: with_capacity_in<u8, alloc::alloc::Global> (raw_vec.rs:130)
==309525== by 0x575E9EB: with_capacity_in<u8, alloc::alloc::Global> (mod.rs:672)
==309525== by 0x575E9EB: <T as alloc::slice::hack::ConvertVec>::to_vec (slice.rs:162)
==309525== by 0x4E9F84B: to_vec<u8, alloc::alloc::Global> (slice.rs:111)
==309525== by 0x4E9F84B: to_vec_in<u8, alloc::alloc::Global> (slice.rs:441)
==309525== by 0x4E9F84B: to_vec<u8> (slice.rs:416)
==309525== by 0x4E9F84B: to_owned<u8> (slice.rs:823)
==309525== by 0x4E9F84B: to_owned (str.rs:209)
==309525== by 0x4E9F84B: from (string.rs:2667)
==309525== by 0x4E9F84B: <str as alloc::string::ToString>::to_string (string.rs:2604)
==309525== by 0x53A7CBF: opendal::raw::oio::entry::Entry::new (entry.rs:37)
==309525== by 0x54BAE78: opendal::raw::adapters::typed_kv::backend::KvPager::inner_next_page::{{closure}} (backend.rs:339)
==309525== by 0x53B1DD1: core::iter::adapters::map::map_fold::{{closure}} (map.rs:84)
==309525== by 0x52DBC4A: core::iter::traits::iterator::Iterator::fold (iterator.rs:2481)
==309525== by 0x53AAC23: <core::iter::adapters::map::Map<I,F> as core::iter::traits::iterator::Iterator>::fold (map.rs:124)
==309525== by 0x53AD95E: core::iter::traits::iterator::Iterator::for_each (iterator.rs:856)
==309525==
[ OK ] OpendalListTest.ListDirTest (2160 ms)
[ RUN ] OpendalListTest.ListEmptyDirTest
[ OK ] OpendalListTest.ListEmptyDirTest (1 ms)
[ RUN ] OpendalListTest.ListNotExistDirTest
[ OK ] OpendalListTest.ListNotExistDirTest (0 ms)
[----------] 3 tests from OpendalListTest (2165 ms total)
[----------] Global test environment tear-down
[==========] 3 tests from 1 test suite ran. (2186 ms total)
[ PASSED ] 3 tests.
==309525==
==309525== HEAP SUMMARY:
==309525== in use at exit: 577 bytes in 4 blocks
==309525== total heap usage: 350 allocs, 346 frees, 8,514,401 bytes allocated
==309525==
==309525== 248 (8 direct, 240 indirect) bytes in 1 blocks are definitely lost in loss record 3 of 4
==309525== at 0x4841848: malloc (vg_replace_malloc.c:431)
==309525== by 0x4E3592D: alloc (alloc.rs:102)
==309525== by 0x4E3592D: alloc::alloc::Global::alloc_impl (alloc.rs:185)
==309525== by 0x4E356D6: allocate (alloc.rs:245)
==309525== by 0x4E356D6: alloc::alloc::exchange_malloc (alloc.rs:334)
==309525== by 0x4E3476A: new<opendal_c::types::opendal_metadata> (boxed.rs:217)
==309525== by 0x4E3476A: opendal_operator_stat (lib.rs:419)
==309525== by 0x10BA63: OpendalListTest_ListDirTest_Test::TestBody() (in /home/xuanwo/Code/apache/incubator-opendal/bindings/c/build/list)
==309525== by 0x5FDCB3B: UnknownInlinedFun (gtest.cc:2621)
==309525== by 0x5FDCB3B: void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) [clone .constprop.0] (gtest.cc:2657)
==309525== by 0x5FC9197: UnknownInlinedFun (gtest.cc:2696)
==309525== by 0x5FC9197: testing::Test::Run() (gtest.cc:2686)
==309525== by 0x5FC93A6: testing::TestInfo::Run() (gtest.cc:2845)
==309525== by 0x5FC9503: UnknownInlinedFun (gtest.cc:3004)
==309525== by 0x5FC9503: testing::TestSuite::Run() (gtest.cc:2977)
==309525== by 0x5FD5A6E: testing::internal::UnitTestImpl::RunAllTests() (gtest.cc:5890)
==309525== by 0x5FD46A8: UnknownInlinedFun (gtest.cc:2621)
==309525== by 0x5FD46A8: UnknownInlinedFun (gtest.cc:2657)
==309525== by 0x5FD46A8: testing::UnitTest::Run() (gtest.cc:5455)
==309525== by 0x10CAC3: RUN_ALL_TESTS() (in /home/xuanwo/Code/apache/incubator-opendal/bindings/c/build/list)
==309525==
==309525== 329 (264 direct, 65 indirect) bytes in 1 blocks are definitely lost in loss record 4 of 4
==309525== at 0x4841848: malloc (vg_replace_malloc.c:431)
==309525== by 0x4E3592D: alloc (alloc.rs:102)
==309525== by 0x4E3592D: alloc::alloc::Global::alloc_impl (alloc.rs:185)
==309525== by 0x4E356D6: allocate (alloc.rs:245)
==309525== by 0x4E356D6: alloc::alloc::exchange_malloc (alloc.rs:334)
==309525== by 0x4E333E6: new<opendal::types::entry::Entry> (boxed.rs:217)
==309525== by 0x4E333E6: opendal_c::types::opendal_list_entry::new (types.rs:373)
==309525== by 0x4E3319C: opendal_lister_next (types.rs:343)
==309525== by 0x10BA24: OpendalListTest_ListDirTest_Test::TestBody() (in /home/xuanwo/Code/apache/incubator-opendal/bindings/c/build/list)
==309525== by 0x5FDCB3B: UnknownInlinedFun (gtest.cc:2621)
==309525== by 0x5FDCB3B: void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) [clone .constprop.0] (gtest.cc:2657)
==309525== by 0x5FC9197: UnknownInlinedFun (gtest.cc:2696)
==309525== by 0x5FC9197: testing::Test::Run() (gtest.cc:2686)
==309525== by 0x5FC93A6: testing::TestInfo::Run() (gtest.cc:2845)
==309525== by 0x5FC9503: UnknownInlinedFun (gtest.cc:3004)
==309525== by 0x5FC9503: testing::TestSuite::Run() (gtest.cc:2977)
==309525== by 0x5FD5A6E: testing::internal::UnitTestImpl::RunAllTests() (gtest.cc:5890)
==309525== by 0x5FD46A8: UnknownInlinedFun (gtest.cc:2621)
==309525== by 0x5FD46A8: UnknownInlinedFun (gtest.cc:2657)
==309525== by 0x5FD46A8: testing::UnitTest::Run() (gtest.cc:5455)
==309525==
==309525== LEAK SUMMARY:
==309525== definitely lost: 272 bytes in 2 blocks
==309525== indirectly lost: 305 bytes in 2 blocks
==309525== possibly lost: 0 bytes in 0 blocks
==309525== still reachable: 0 bytes in 0 blocks
==309525== suppressed: 0 bytes in 0 blocks
==309525==
==309525== For lists of detected and suppressed errors, rerun with: -s
==309525== ERROR SUMMARY: 4 errors from 4 contexts (suppressed: 0 from 0)
make: *** [Makefile:48: test] Error 1 |
Xuanwo
changed the title
Contributor
|
Just some update, my libc was messed up today, I am rebuilding the system and libraries. So I think the fix should come later tomorrow. |
Contributor
|
Hi @wildbartty , I have raised the fix in #2673. I believe this could be merged as soon as that one get merged. |
Member
|
#2673 merged! |
Member
|
Hey @wildbartty, do you want to |
Contributor
Author
|
@Xuanwo I've updated my branch. |
xyjixyjixyji
approved these changes
Jul 20, 2023
Contributor
xyjixyjixyji
left a comment
xyjixyjixyji
left a comment
There was a problem hiding this comment.
Thanks for the work! The PR LGTM. Let's wait the CI to pass.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is for issue #2495
Note that it currently breaks the ci testing because it has found memory leaks