Conversation
…preallocates one read-only transaction and reuses it. It's very important that iterations are considered invalid after a commit has been performed.
|
This patch seems to fix some failed database unit tests (from #1238). Without this patch, I get an error in the "dynamic" linked tests. It happens in multiple tests. One example error that happens if you use #1238 but not #1319: [ RUN ] DataLayerTest/1.TestReadCropTrainSequenceUnseededLMDB
I1015 19:14:43.326297 18597 test_data_layer.cpp:41] Using temporary dataset /tmp/caffe_test.8SJB3R/db
I1015 19:14:43.328063 18597 data_layer.cpp:31] Opening dataset /tmp/caffe_test.8SJB3R/db
*** Aborted at 1413414883 (unix time) try "date -d @1413414883" if you are using GNU date ***
PC: @ 0x2b73849ed8c7 __pthread_mutex_lock_full
*** SIGSEGV (@0x2b7390cda058) received by PID 18597 (TID 0x2b73907bdb00) from PID 18446744071843979352; stack trace: ***
@ 0x2b73849f7cb0 (unknown)
@ 0x2b73849ed8c7 __pthread_mutex_lock_full
@ 0x2b73829502a6 mdb_txn_renew0
@ 0x2b73829512ef mdb_txn_begin
@ 0x6dd56d caffe::LmdbDataset<>::open()
@ 0x76e1f2 caffe::DataLayer<>::DataLayerSetUp()
@ 0x77f5b3 caffe::BaseDataLayer<>::LayerSetUp()
@ 0x77f5c9 caffe::BasePrefetchingDataLayer<>::LayerSetUp()
@ 0x5e21d8 caffe::DataLayerTest<>::TestReadCropTrainSequenceUnseeded()
@ 0x69266d testing::internal::HandleExceptionsInMethodIfSupported<>()
@ 0x68a441 testing::Test::Run()
@ 0x68a526 testing::TestInfo::Run()
@ 0x68a667 testing::TestCase::Run()
@ 0x68a9be testing::internal::UnitTestImpl::RunAllTests()
@ 0x6921ed testing::internal::HandleExceptionsInMethodIfSupported<>()
@ 0x689a9e testing::UnitTest::Run()
@ 0x46ddd0 main
@ 0x2b7384c2676d (unknown)
@ 0x4738bd (unknown)
Segmentation fault (core dumped)
make: *** [runtest] Error 139
|
Member
|
Merging as partial fix while the full fix and interface simplification is done according to the plan in the #1238 post-merge discussion. |
shelhamer
added a commit
that referenced
this pull request
Oct 19, 2014
LMDB read-only transaction limit fix
RazvanRanca
pushed a commit
to RazvanRanca/caffe
that referenced
this pull request
Nov 4, 2014
LMDB read-only transaction limit fix
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I incorrectly assumed that you could have many concurrent read-only transactions with LMDB. This preallocates a single read-only transaction that can be reused until commit has been called.
This still leaks cursors. I need to find out whether or not those are automatically cleaned up when a transaction aborts or commits.
New test cases included.