Allow to write empty datasets by franzpoeschel · Pull Request #529 · openPMD/openPMD-api

franzpoeschel · 2019-06-20T11:52:36Z

Close #514 and #516.
This PR adds the following member to the class RecordComponent:

    template< typename T >
    RecordComponent& makeEmpty( uint8_t dimensions );

This lets the user explicitly create an empty dataset.
Upon detecting an empty dataset, the member RecordComponent& resetDataset(Dataset); will not fail any more, but also create an empty dataset.

Since backends do currently not expect empty datasets, I implemented this feature by using constant record components. Zero-dimensional datasets are still not allowed.

The arguments are sanity-checked, but no operation is passed down to the backend.

See openPMD PR: openPMD/openPMD-api#529 This allows to write empty datasets

ax3l

Just a few little inline comments.

Can you please expose the new makeEmpty functionality in the Python bindings as well? :)

include/openPMD/RecordComponent.hpp

    RecordComponent& makeConstant(T);

+    template< typename T >
+    RecordComponent& makeEmpty( uint8_t dimensions );


ax3l · 2019-06-26T18:14:00Z

include/openPMD/RecordComponent.hpp


    std::shared_ptr< std::queue< IOTask > > m_chunks;
    std::shared_ptr< Attribute > m_constantValue;
+    std::shared_ptr< bool > m_isEmpty{


shouldn't a simple = ... work as well here?

include/openPMD/RecordComponent.hpp

 private:
    void flush(std::string const&);
    virtual void read();
+    RecordComponent& makeEmpty( Dataset );


ax3l · 2019-06-26T18:16:26Z

include/openPMD/RecordComponent.hpp

 RecordComponent::storeChunk(std::shared_ptr<T> data, Offset o, Extent e)
 {
-    if( *m_isConstant )
+    if( *m_isConstant && !*m_isEmpty )


shouldn't we throw a runtime error on m_isEmpty if storeChunk is called on it?

My intention was to allow writing generic code that does not need to distinguish empty datasets from non-empty ones. The offset and extent are still sanity-checked, meaning that the call to storeChunk will throw anyway if a non-empty chunk is passed. If all checks pass, this operation is a no-op.

So just to understand this better: what do you consider a valid value for data in this case?

I guess you are hinting at the following lines?

if( !data ) throw std::runtime_error("Unallocated pointer passed during chunk store.");

You are raising a point that I didn't think about thoroughly enough apparently. C++ does not allow zero-sized arrays (I was under the impression it did), so the user will need to make some distinction. (Meaning I'll have to check my PIConGPU plugin)
I guess, we should let nullpointers slide in this case?

I mean I follow your wish to have simple code paths, but just contextually it seams wrong to me to call storeChunk on a constant or empty record component.

If we decided to allow this, which I feel makes the API actually more corner-casy than simple, then the only logically allowed value for data should be a nullptr because everything else is quite surprising.

Using storeChunk on a constant record component is still forbidden because its intention in the openPMD standard is to be an alternative to dealing with chunks.
Here, on the other hand, constant record components are merely an (arguably quick and dirty) implementation to allow storing chunks that are incidentally empty.

But given this implementation, I see your point that storeChunk may give the wrong impression that the backend will eventually receive a dataset-related task which is definitely not the case. And since the C++ standard already makes a unified code path impossible, I would be ok with removing this functionality again if you prefer.

Related question: storeChunk does not yet support workflows where one rank contributes no data to a dataset that has data, right? We should consider this, if we decide to keep the change.

(answered below since mail answer was wrapped around)

include/openPMD/backend/BaseRecordComponent.hpp

+namespace detail
+{
+template< typename RecordComponent_T >
+struct DefaultValue


include/openPMD/backend/BaseRecordComponent.hpp

    std::shared_ptr< Dataset > m_dataset;
    std::shared_ptr< bool > m_isConstant;
 }; // BaseRecordComponent
+namespace detail


ax3l · 2019-06-26T18:19:00Z

src/RecordComponent.cpp

 {
    if( written )
-        throw std::runtime_error("A Records Dataset can not (yet) be changed after it has been written.");
+        throw std::runtime_error( "A Records Dataset can not (yet) be changed "


typos:

Suggested change

throw std::runtime_error( "A Records Dataset can not (yet) be changed "

throw std::runtime_error( "A record's Dataset cannot (yet) be changed "

ax3l · 2019-06-26T18:19:34Z

src/RecordComponent.cpp

+{
+    if( written )
+        throw std::runtime_error(
+            "A recordComponent can not (yet) be made"


RecordComponent is a type, so let's keep the spelling to the typename as above:

Suggested change

"A recordComponent can not (yet) be made"

"A RecordComponent cannot (yet) be made"

ax3l · 2019-06-26T18:20:37Z

test/SerialIOTest.cpp

+{
+    for (auto const & t: backends)
+    {
+        //if (std::get<0>(t) == "json")


Add documentation, change some typos in error messages, clean dead code

ax3l · 2019-06-27T13:56:05Z

Agreed.

Related question: `storeChunk` does not yet support workflows where one rank contributes no data to a dataset that has data, right? We should consider this, if we decide to keep the change.

Regarding the last point: no, storeChunk supports partial zero contributions (if not even all-zero for most backends). That's an important use case, e.g. for particles.

franzpoeschel · 2019-06-27T13:59:34Z

Agreed = agreed to removing it again?

storeChunk supports partial zero contributions (if not even all-zero for most backends)

Isn't the nullpointer check problematic in this case?

ax3l · 2019-06-30T11:07:18Z

include/openPMD/RecordComponent.hpp


+    /**
+     * Create a dataset with zero extent in each dimension.
+     * Implemented by creating a constant record component with


potentially add an empty comment line to separate title and brief description in doxygen :)

ax3l · 2019-06-30T11:08:41Z

include/openPMD/backend/BaseRecordComponent.hpp

+namespace detail
+{
+/**
+ * Functor template to be used in combination with <switchType>"()"


Maybe also keep a one-line title and a separate brief description here.

ax3l · 2019-06-30T11:10:37Z

Isn't the nullpointer check problematic in this case?

No, no contribution means: do not call storeChunk, as it already is.

See also openPMD#529

See also openPMD#529 Add a non-template overload to RecordComponent::makeEmpty Add tests for previous commit Refactor python binding to use non-template version of makeEmpty Add python tests for make_empty Add an overload to RecordComponent.make_empty for numpy datatypes Add tests for numpy datatypes Fix datatype tests Only test fixed-sized types

See openPMD PR: openPMD/openPMD-api#529 This allows to write empty datasets

Create openPMDWriter based on ADIOSWriter (WIP) Adapt CountParticles partially to openPMD Adapt WriteSpecies to openPMD Translate WriteMeta.hpp to openPMD Add function "asStandardVector()" Translate NDScalars.hpp to openPMD Adapt everything to openPMD except for the main file (openPMDWriter.hpp) Adapt openPMDWriter.hpp to openPMD WIP Change management of openPMD Series Further adapt openPMDWriter to openPMD Add openPMD to CMake Add openPMD to version formatting Properly acquaint openPMD plugin with build and help system Make openPMD namespacing explicit Remove adios-specific fields from ThreadParams First successfull run with LaserWakeField example Cleanup Use clang-format Update licensing information Separate basename and filename extension into two separate help parameters Refactor dataset creation Use Mesh::setTimeOffset() template Causes a linker error with the current version of openPMD API, a fix will be merged soon on the dev branch. Clean up some leftovers Use ASCII characters to spell my name Remove unnecessary whitespaces Adapt to removal of pmacc::forward Remove accidentally versioned config file Make checkpoints usable Fix a number of bugs concerning the reading of checkpoints Still problematic is the attribute "timeOffset", currently mitigated by uncommenting the sanity check in the openPMD API. Needs further investigation. Remove CollectFieldsSizes Legacy from ADIOS writer, not necessary in openPMD. Remove ParticleAttributeSize and openPMDCountParticles Legacy from ADIOS Writer Use clang-format Adhere to openPMD requirements before flushing For a given particle species, the openPMD API requires that required records (such as "position", "positionOffset") and their contained components be defined (not necessarily written). Make sure to define all such datasets before issuing the first flush. Maybe open an issue in the openPMD API to allow for a more flexible usage. Fix an indexing bug Eliminate dataset preparation for NDScalars Also fix a bug where particles were named wrongly in checkpoints. Do not write empty particle datasets Treat non-existing datasets as empty in reading Remove prepared datasets Remove WithWindow struct Use transform to enable ADIOS1 compression Remove accidentally versioned files Rename LoadParticleAttributesFromADIOS to LoadParticleAttributesFromOpenPMD Remove traces of the old ADIOS plugin mostly the word "adios" from comments Take copies and not references of openPMD handles Fix autoformatting Require newer openPMD API Also add ADIOS2_ROOT to CMAKE_PREFIX_PATH Add version label to format string only if present Replace typedefs with using Remove further indexing bug in writing particles_info table Cleanup restart Remove dataset preparation Commit 0b50561 reintroduced a reduced form of dataset preparation in order to adhere to requirements (restrictions) of the openPMD API. This workaround results in dummy dataset writes (likely a bug in the openPMD API), hence this commit reverts those changes. The corresponding pull request in the openPMD API to relax this restriction can be found at openPMD/openPMD-api#518. Postpone writing of meta attributes Due to a bug in the ADIOS1 backend of the openPMD API, the first dataset needs to be written before the first flush. This works around the problem by first defining all the fields and particles. Bug report will follow. Resolve counting bug during particle writing Fix whitespaces Separate ADIOS and HDF5 particle writing strategies Allow choosing strategy as runtime parameter Cleanup Fix openPMD version formatting Update examples to use openPMD Refactor passing of filename and extension Reassemble filename and extension upon opening the series. Fix some missing includes Do not skip writing empty datasets See openPMD PR: openPMD/openPMD-api#529 This allows to write empty datasets Remove debugging leftovers Write timeOffset for particles in a standard-compliant way Do not declare zero-sized arrays C++ standard requires that array size evaluate to a value greater than zero. Do not use storeChunk on empty datasets centralize initialization of thread params from config Fix undefined identifier in assert statements Error passes silently in release builds. Pass JSON config to openPMD::Series ctor Do not copy Series object see openPMD/openPMD-api#534 Allow NULL as configuration infix to denote empty string Adapt to changes in pmacc etc. Enable use of group-based layout as well Requires keeping openPMD Series open. Since openPMD currently has no explicit call to close a file, we implement this only for group-based layout for now. Do not use deprecated Series::setSoftwareVersion call Apply commit b797c0a to openPMD backend Formatting in .cfg files Fix an uninitialized value and an indexing bug Implement reviewers' comments concerning CMakeLists

See also openPMD#529 Add a non-template overload to RecordComponent::makeEmpty Add tests for previous commit Refactor python binding to use non-template version of makeEmpty Add python tests for make_empty Add an overload to RecordComponent.make_empty for numpy datatypes Add tests for numpy datatypes Fix datatype tests Only test fixed-sized types

Create openPMDWriter based on ADIOSWriter (WIP) Adapt CountParticles partially to openPMD Adapt WriteSpecies to openPMD Translate WriteMeta.hpp to openPMD Add function "asStandardVector()" Translate NDScalars.hpp to openPMD Adapt everything to openPMD except for the main file (openPMDWriter.hpp) Adapt openPMDWriter.hpp to openPMD WIP Change management of openPMD Series Further adapt openPMDWriter to openPMD Add openPMD to CMake Add openPMD to version formatting Properly acquaint openPMD plugin with build and help system Make openPMD namespacing explicit Remove adios-specific fields from ThreadParams First successfull run with LaserWakeField example Cleanup Use clang-format Update licensing information Separate basename and filename extension into two separate help parameters Refactor dataset creation Use Mesh::setTimeOffset() template Causes a linker error with the current version of openPMD API, a fix will be merged soon on the dev branch. Clean up some leftovers Use ASCII characters to spell my name Remove unnecessary whitespaces Adapt to removal of pmacc::forward Remove accidentally versioned config file Make checkpoints usable Fix a number of bugs concerning the reading of checkpoints Still problematic is the attribute "timeOffset", currently mitigated by uncommenting the sanity check in the openPMD API. Needs further investigation. Remove CollectFieldsSizes Legacy from ADIOS writer, not necessary in openPMD. Remove ParticleAttributeSize and openPMDCountParticles Legacy from ADIOS Writer Use clang-format Adhere to openPMD requirements before flushing For a given particle species, the openPMD API requires that required records (such as "position", "positionOffset") and their contained components be defined (not necessarily written). Make sure to define all such datasets before issuing the first flush. Maybe open an issue in the openPMD API to allow for a more flexible usage. Fix an indexing bug Eliminate dataset preparation for NDScalars Also fix a bug where particles were named wrongly in checkpoints. Do not write empty particle datasets Treat non-existing datasets as empty in reading Remove prepared datasets Remove WithWindow struct Use transform to enable ADIOS1 compression Remove accidentally versioned files Rename LoadParticleAttributesFromADIOS to LoadParticleAttributesFromOpenPMD Remove traces of the old ADIOS plugin mostly the word "adios" from comments Take copies and not references of openPMD handles Fix autoformatting Require newer openPMD API Also add ADIOS2_ROOT to CMAKE_PREFIX_PATH Add version label to format string only if present Replace typedefs with using Remove further indexing bug in writing particles_info table Cleanup restart Remove dataset preparation Commit 0b50561 reintroduced a reduced form of dataset preparation in order to adhere to requirements (restrictions) of the openPMD API. This workaround results in dummy dataset writes (likely a bug in the openPMD API), hence this commit reverts those changes. The corresponding pull request in the openPMD API to relax this restriction can be found at openPMD/openPMD-api#518. Postpone writing of meta attributes Due to a bug in the ADIOS1 backend of the openPMD API, the first dataset needs to be written before the first flush. This works around the problem by first defining all the fields and particles. Bug report will follow. Resolve counting bug during particle writing Fix whitespaces Separate ADIOS and HDF5 particle writing strategies Allow choosing strategy as runtime parameter Cleanup Fix openPMD version formatting Update examples to use openPMD Refactor passing of filename and extension Reassemble filename and extension upon opening the series. Fix some missing includes Do not skip writing empty datasets See openPMD PR: openPMD/openPMD-api#529 This allows to write empty datasets Remove debugging leftovers Write timeOffset for particles in a standard-compliant way Do not declare zero-sized arrays C++ standard requires that array size evaluate to a value greater than zero. Do not use storeChunk on empty datasets centralize initialization of thread params from config Fix undefined identifier in assert statements Error passes silently in release builds. Pass JSON config to openPMD::Series ctor Do not copy Series object see openPMD/openPMD-api#534 Allow NULL as configuration infix to denote empty string Adapt to changes in pmacc etc. Enable use of group-based layout as well Requires keeping openPMD Series open. Since openPMD currently has no explicit call to close a file, we implement this only for group-based layout for now. Do not use deprecated Series::setSoftwareVersion call Apply commit b797c0a to openPMD backend Formatting in .cfg files Fix an uninitialized value and an indexing bug Implement reviewers' comments concerning CMakeLists

See also openPMD#529 Add a non-template overload to RecordComponent::makeEmpty Add tests for previous commit Refactor python binding to use non-template version of makeEmpty Add python tests for make_empty Add an overload to RecordComponent.make_empty for numpy datatypes Add tests for numpy datatypes Fix datatype tests Only test fixed-sized types

Create openPMDWriter based on ADIOSWriter (WIP) Adapt CountParticles partially to openPMD Adapt WriteSpecies to openPMD Translate WriteMeta.hpp to openPMD Add function "asStandardVector()" Translate NDScalars.hpp to openPMD Adapt everything to openPMD except for the main file (openPMDWriter.hpp) Adapt openPMDWriter.hpp to openPMD WIP Change management of openPMD Series Further adapt openPMDWriter to openPMD Add openPMD to CMake Add openPMD to version formatting Properly acquaint openPMD plugin with build and help system Make openPMD namespacing explicit Remove adios-specific fields from ThreadParams First successfull run with LaserWakeField example Cleanup Use clang-format Update licensing information Separate basename and filename extension into two separate help parameters Refactor dataset creation Use Mesh::setTimeOffset() template Causes a linker error with the current version of openPMD API, a fix will be merged soon on the dev branch. Clean up some leftovers Use ASCII characters to spell my name Remove unnecessary whitespaces Adapt to removal of pmacc::forward Remove accidentally versioned config file Make checkpoints usable Fix a number of bugs concerning the reading of checkpoints Still problematic is the attribute "timeOffset", currently mitigated by uncommenting the sanity check in the openPMD API. Needs further investigation. Remove CollectFieldsSizes Legacy from ADIOS writer, not necessary in openPMD. Remove ParticleAttributeSize and openPMDCountParticles Legacy from ADIOS Writer Use clang-format Adhere to openPMD requirements before flushing For a given particle species, the openPMD API requires that required records (such as "position", "positionOffset") and their contained components be defined (not necessarily written). Make sure to define all such datasets before issuing the first flush. Maybe open an issue in the openPMD API to allow for a more flexible usage. Fix an indexing bug Eliminate dataset preparation for NDScalars Also fix a bug where particles were named wrongly in checkpoints. Do not write empty particle datasets Treat non-existing datasets as empty in reading Remove prepared datasets Remove WithWindow struct Use transform to enable ADIOS1 compression Remove accidentally versioned files Rename LoadParticleAttributesFromADIOS to LoadParticleAttributesFromOpenPMD Remove traces of the old ADIOS plugin mostly the word "adios" from comments Take copies and not references of openPMD handles Fix autoformatting Require newer openPMD API Also add ADIOS2_ROOT to CMAKE_PREFIX_PATH Add version label to format string only if present Replace typedefs with using Remove further indexing bug in writing particles_info table Cleanup restart Remove dataset preparation Commit 0b50561 reintroduced a reduced form of dataset preparation in order to adhere to requirements (restrictions) of the openPMD API. This workaround results in dummy dataset writes (likely a bug in the openPMD API), hence this commit reverts those changes. The corresponding pull request in the openPMD API to relax this restriction can be found at openPMD/openPMD-api#518. Postpone writing of meta attributes Due to a bug in the ADIOS1 backend of the openPMD API, the first dataset needs to be written before the first flush. This works around the problem by first defining all the fields and particles. Bug report will follow. Resolve counting bug during particle writing Fix whitespaces Separate ADIOS and HDF5 particle writing strategies Allow choosing strategy as runtime parameter Cleanup Fix openPMD version formatting Update examples to use openPMD Refactor passing of filename and extension Reassemble filename and extension upon opening the series. Fix some missing includes Do not skip writing empty datasets See openPMD PR: openPMD/openPMD-api#529 This allows to write empty datasets Remove debugging leftovers Write timeOffset for particles in a standard-compliant way Do not declare zero-sized arrays C++ standard requires that array size evaluate to a value greater than zero. Do not use storeChunk on empty datasets centralize initialization of thread params from config Fix undefined identifier in assert statements Error passes silently in release builds. Pass JSON config to openPMD::Series ctor Do not copy Series object see openPMD/openPMD-api#534 Allow NULL as configuration infix to denote empty string Adapt to changes in pmacc etc. Enable use of group-based layout as well Requires keeping openPMD Series open. Since openPMD currently has no explicit call to close a file, we implement this only for group-based layout for now. Do not use deprecated Series::setSoftwareVersion call Apply commit b797c0a to openPMD backend Formatting in .cfg files Fix an uninitialized value and an indexing bug Implement reviewers' comments concerning CMakeLists

Create openPMDWriter based on ADIOSWriter (WIP) Adapt CountParticles partially to openPMD Adapt WriteSpecies to openPMD Translate WriteMeta.hpp to openPMD Add function "asStandardVector()" Translate NDScalars.hpp to openPMD Adapt everything to openPMD except for the main file (openPMDWriter.hpp) Adapt openPMDWriter.hpp to openPMD WIP Change management of openPMD Series Further adapt openPMDWriter to openPMD Add openPMD to CMake Add openPMD to version formatting Properly acquaint openPMD plugin with build and help system Make openPMD namespacing explicit Remove adios-specific fields from ThreadParams First successfull run with LaserWakeField example Cleanup Use clang-format Update licensing information Separate basename and filename extension into two separate help parameters Refactor dataset creation Use Mesh::setTimeOffset() template Causes a linker error with the current version of openPMD API, a fix will be merged soon on the dev branch. Clean up some leftovers Use ASCII characters to spell my name Remove unnecessary whitespaces Adapt to removal of pmacc::forward Remove accidentally versioned config file Make checkpoints usable Fix a number of bugs concerning the reading of checkpoints Still problematic is the attribute "timeOffset", currently mitigated by uncommenting the sanity check in the openPMD API. Needs further investigation. Remove CollectFieldsSizes Legacy from ADIOS writer, not necessary in openPMD. Remove ParticleAttributeSize and openPMDCountParticles Legacy from ADIOS Writer Use clang-format Adhere to openPMD requirements before flushing For a given particle species, the openPMD API requires that required records (such as "position", "positionOffset") and their contained components be defined (not necessarily written). Make sure to define all such datasets before issuing the first flush. Maybe open an issue in the openPMD API to allow for a more flexible usage. Fix an indexing bug Eliminate dataset preparation for NDScalars Also fix a bug where particles were named wrongly in checkpoints. Do not write empty particle datasets Treat non-existing datasets as empty in reading Remove prepared datasets Remove WithWindow struct Use transform to enable ADIOS1 compression Remove accidentally versioned files Rename LoadParticleAttributesFromADIOS to LoadParticleAttributesFromOpenPMD Remove traces of the old ADIOS plugin mostly the word "adios" from comments Take copies and not references of openPMD handles Fix autoformatting Require newer openPMD API Also add ADIOS2_ROOT to CMAKE_PREFIX_PATH Add version label to format string only if present Replace typedefs with using Remove further indexing bug in writing particles_info table Cleanup restart Remove dataset preparation Commit 0b50561 reintroduced a reduced form of dataset preparation in order to adhere to requirements (restrictions) of the openPMD API. This workaround results in dummy dataset writes (likely a bug in the openPMD API), hence this commit reverts those changes. The corresponding pull request in the openPMD API to relax this restriction can be found at openPMD/openPMD-api#518. Postpone writing of meta attributes Due to a bug in the ADIOS1 backend of the openPMD API, the first dataset needs to be written before the first flush. This works around the problem by first defining all the fields and particles. Bug report will follow. Resolve counting bug during particle writing Fix whitespaces Separate ADIOS and HDF5 particle writing strategies Allow choosing strategy as runtime parameter Cleanup Fix openPMD version formatting Update examples to use openPMD Refactor passing of filename and extension Reassemble filename and extension upon opening the series. Fix some missing includes Do not skip writing empty datasets See openPMD PR: openPMD/openPMD-api#529 This allows to write empty datasets Remove debugging leftovers Write timeOffset for particles in a standard-compliant way Do not declare zero-sized arrays C++ standard requires that array size evaluate to a value greater than zero. Do not use storeChunk on empty datasets centralize initialization of thread params from config Fix undefined identifier in assert statements Error passes silently in release builds. Pass JSON config to openPMD::Series ctor Do not copy Series object see openPMD/openPMD-api#534 Allow NULL as configuration infix to denote empty string Adapt to changes in pmacc etc. Enable use of group-based layout as well Requires keeping openPMD Series open. Since openPMD currently has no explicit call to close a file, we implement this only for group-based layout for now. Do not use deprecated Series::setSoftwareVersion call Apply commit b797c0a to openPMD backend Formatting in .cfg files Fix an uninitialized value and an indexing bug Implement reviewers' comments concerning CMakeLists Fix some bugs and remove unneeded parameters Some ADIOS-specific parameters have been remove that haven't been implemented anyway and are going to be implemented via JSON. Don't set compression if set to "none". Also, I missed a small part when porting b797c0a to this plugin. Add documentation for openPMD plugin [WIP] Further write documentation Rename data preparation strategies adios -> doubleBuffer hdf5 -> mappedMemory Old names may still be used alternatively. Fix indexing bug for non domain-bound fields Fix indexing order in fields Implement Reviewer's comments Remove workaround for ADIOS1 backend fix attributes Mesh record component positions and unitSI Iteration: time ParticleAttribute unitDimension Implement reviewer's comments (2) Update copyright headers of files changed Add compression to openPMD TBG example Use openPMD::getVersion remove unnecessary parentheses

See also openPMD#529 Add a non-template overload to RecordComponent::makeEmpty Add tests for previous commit Refactor python binding to use non-template version of makeEmpty Add python tests for make_empty Add an overload to RecordComponent.make_empty for numpy datatypes Add tests for numpy datatypes Fix datatype tests Only test fixed-sized types

* Add Python bindings for RecordComponent::makeEmpty() See also #529 Add a non-template overload to RecordComponent::makeEmpty Add tests for previous commit Refactor python binding to use non-template version of makeEmpty Add python tests for make_empty Add an overload to RecordComponent.make_empty for numpy datatypes Add tests for numpy datatypes Fix datatype tests Only test fixed-sized types * Fix whitespaces

franzpoeschel added 5 commits June 20, 2019 13:45

Implement makeEmpty for record components

5ebd5dc

Add empty_dataset_test

77c568e

Disallow zero-dimensional datasets again

17c9ebc

Allow zero-dimensional calls to RecordComponent::makeEmpty<T>()

8488849

Fix object slicing in tests

a9f8737

franzpoeschel force-pushed the topic-empty-datasets branch from 5ceadfe to a9f8737 Compare June 21, 2019 08:17

Allow to storeChunk on an empty dataset

3de8a3c

The arguments are sanity-checked, but no operation is passed down to the backend.

franzpoeschel added a commit to franzpoeschel/picongpu that referenced this pull request Jun 21, 2019

Do not skip writing empty datasets

51d55c9

See openPMD PR: openPMD/openPMD-api#529 This allows to write empty datasets

franzpoeschel mentioned this pull request Jun 21, 2019

openPMD Plugin ComputationalRadiationPhysics/picongpu#2966

Merged

10 tasks

ax3l added api: new additions to the API frontend: C++17 labels Jun 26, 2019

ax3l self-requested a review June 26, 2019 18:12

ax3l reviewed Jun 26, 2019

View reviewed changes

franzpoeschel added 2 commits June 27, 2019 11:21

Cleanup

2aa0765

Add documentation, change some typos in error messages, clean dead code

Require nullpointer on chunk store of an empty dataset

34e2aa7

Disallow the usage of storeChunk on empty record components

3adee76

ax3l reviewed Jun 30, 2019

View reviewed changes

ax3l approved these changes Jun 30, 2019

View reviewed changes

ax3l merged commit 42f4340 into openPMD:dev Jun 30, 2019

franzpoeschel added a commit to franzpoeschel/openPMD-api that referenced this pull request Jul 25, 2019

Add Python bindings for RecordComponent::makeEmpty()

02e238c

See also openPMD#529

franzpoeschel mentioned this pull request Jul 25, 2019

Record_Component.make_empty() #538

Merged

franzpoeschel added a commit to franzpoeschel/openPMD-api that referenced this pull request Aug 7, 2019

Add Python bindings for RecordComponent::makeEmpty()

6e382b9

See also openPMD#529

franzpoeschel added a commit to franzpoeschel/picongpu that referenced this pull request Mar 4, 2020

Do not skip writing empty datasets

f24ba0b

See openPMD PR: openPMD/openPMD-api#529 This allows to write empty datasets

ax3l mentioned this pull request Dec 17, 2020

compatibility issues of new openpmd-api openPMD/openPMD-viewer#296

Closed

franzpoeschel deleted the topic-empty-datasets branch January 28, 2021 13:26

	throw std::runtime_error( "A Records Dataset can not (yet) be changed "
	throw std::runtime_error( "A record's Dataset cannot (yet) be changed "

	"A recordComponent can not (yet) be made"
	"A RecordComponent cannot (yet) be made"

Conversation

franzpoeschel commented Jun 20, 2019

Uh oh!

ax3l left a comment

Choose a reason for hiding this comment

Uh oh!

This comment was marked as resolved.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This comment was marked as resolved.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ax3l Jun 26, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ax3l commented Jun 27, 2019 via email • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

franzpoeschel commented Jun 27, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ax3l commented Jun 30, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

ax3l Jun 26, 2019 •

edited

Loading

ax3l commented Jun 27, 2019 via email •

edited

Loading

franzpoeschel commented Jun 27, 2019 •

edited

Loading

ax3l commented Jun 30, 2019 •

edited

Loading