Don't read iterations if they have already been parsed#1089
Merged
ax3l merged 1 commit intoopenPMD:devfrom Aug 17, 2021
Merged
Conversation
ax3l
pushed a commit
to ax3l/openPMD-api
that referenced
this pull request
Aug 18, 2021
ax3l
added a commit
that referenced
this pull request
Aug 18, 2021
* Bug fix: Don't forget closing files (#1083) * Failing test * Bug fix: Don't forget closing files * HDF5: Fix String Vlen Attribute Reads (#1084) We inofficially try to also support HDF5 variable lengths strings in reading, just to increase compatibility. This was never working it seems. * Fix reading of vector attributes with only one contained value (#1085) * Failing test * Conversions in Attribute.hpp 1) single values to 1-value vectors 2) vectors to arrays 3) arrays to vectors * Some cleanup in Attribute.hpp 1) Simplify types in DoConvert, remove unnecessary template parameter 2) Replace a long if-then-else chain by variantSrc::visit * CoreTest: Fix std::array constructors Make more widely compile-able. * Explicit casting in some places This avoids some warnings * Intel compilers: Don't use variantSrc::visit They don't like it * Remove scattered checks for vector attributes * Generalize icpc guard As defined in CMake for compiler identification. * Doc ICC version (2021.3.0) * setAttribute: Reject Empty Strings (#1087) * setAttribute: Reject Empty Strings Some backends, especially HDF5, do not allow us to define zero-sized strings. We thus need to catch this in the frontend and forward the restriction to the user. * Test: setAttribute("key", "") throws * setAttribute Check: C++14 compatible * Don't read iterations if they have already been parsed (#1089) * Release: 0.14.2 Co-authored-by: Franz Pöschel <franz.poeschel@gmail.com>
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Until now the condition to not reread an iteration in the streaming API was if the
Iteration.closedattribute was present.In datasets where such an attribute was never written, this leads to reparsing of the entire series in every iteration (i.e. quadratic parsing cost). Iterations are not changed after they are first seen, so rereading is not necessary.