Decay ReadWrite mode into adios2::Mode::Read if the file exists#943
Merged
ax3l merged 3 commits intoopenPMD:devfrom Mar 23, 2021
Merged
Decay ReadWrite mode into adios2::Mode::Read if the file exists#943ax3l merged 3 commits intoopenPMD:devfrom
ax3l merged 3 commits intoopenPMD:devfrom
Conversation
franzpoeschel
force-pushed
the
fix-adios2-accessmode
branch
from
e29b9e8 to
5f11970
Compare
This was referenced Mar 12, 2021
ax3l
reviewed
Mar 17, 2021
| std::cerr << "ADIOS2 does currently not yet implement ReadWrite " | ||
| "(Append) mode. " | ||
| << "Replacing with Write mode." << std::endl; | ||
| return adios2::Mode::Write; |
Member
There was a problem hiding this comment.
Is this safe to keep going instead of asking for another mode?
Contributor
Author
There was a problem hiding this comment.
Choosing read mode is safe anyway and write mode is only chosen if the file does not exist, so it should be safe, too. I can imagine that there might be issues in parallel, I should test how ADIOS2 behaves for that and potentially add an MPI barrier.
Contributor
Author
There was a problem hiding this comment.
According to some quick experiments I did, ADIOS does not create files on the disk if not all of the parallel writers have called Engine::open() yet. So we could leave it as it is or alternatively add an MPI Barrier for some extra safety.
Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja>
franzpoeschel
force-pushed
the
fix-adios2-accessmode
branch
from
efc8c44 to
a64a8bb
Compare
ax3l
approved these changes
Mar 23, 2021
ax3l
added a commit
to ax3l/openPMD-api
that referenced
this pull request
Apr 9, 2021
…PMD#943) * Decay ReadWrite mode into adios2::Mode::Read if the file exists * Add test for read-write mode Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bug detected while working on #855.
EDIT: Alright, found out why this bug was triggered by #855, but not previously. Until now, the ADIOS2 backend delayed the opening of the .bp file until an actual access happened. So this test never actually created the file for iteration 6, bypassing this bug. Now, with eager opening, this bug is exposed.
TODO:
The test exposed the same issue in HDF5. Fix that in this PR too?That was a different issue. I wrote a little mesh dataset to that iteration in the test originally. Now, that iteration had a defined
/meshesPathattribute. While reading the file back in some lines further down, the openPMD API expected to find meshes in the other iterations, too, didn't find any and crashed. Is our workflow with themeshesPathreally that good right now?