Skip to content

Remove dummy dataset writing from RecordComponent::flush()#528

Merged
ax3l merged 2 commits intoopenPMD:devfrom
franzpoeschel:fix-empty-dataset-triggers-write
Jul 1, 2019
Merged

Remove dummy dataset writing from RecordComponent::flush()#528
ax3l merged 2 commits intoopenPMD:devfrom
franzpoeschel:fix-empty-dataset-triggers-write

Conversation

@franzpoeschel
Copy link
Contributor

See #489 and #512 for a description. The exactly same thing happens also in RecordComponent.cpp. PR #512 did not catch this.

@ax3l
Copy link
Member

ax3l commented Jun 17, 2019

CI seams unhappy.
You might want to implement the default for such records and attributes on Series::close() or similar.

@franzpoeschel
Copy link
Contributor Author

franzpoeschel commented Jun 19, 2019

The problem is with the fileBased_write_empty_test which triggers exactly the situation that I have described here:

It is a design decision in ADIOS (also ADIOS2) not to physically create variables (used for openPMD datasets) unless data is actually written and openPMD can leverage this behavior. The workaround removed in this PR tries to circumvent this behavior. Needs to be documented in this case.

Relevant part of the test code:

    Dataset dset = Dataset(Datatype::DOUBLE, {2});
    {
        Series o = Series("../samples/subdir/serial_fileBased_write%T." + backend, AccessType::CREATE);

        ParticleSpecies& e_1 = o.iterations[1].particles["e"];
        e_1["position"][RecordComponent::SCALAR].resetDataset(dset);
        e_1["positionOffset"][RecordComponent::SCALAR].resetDataset(dset);
        o.iterations[1].setTime(1.f);
        ParticleSpecies& e_2 = o.iterations[2].particles["e"];
        e_2["position"][RecordComponent::SCALAR].resetDataset(dset);
        e_2["positionOffset"][RecordComponent::SCALAR].resetDataset(dset);
        o.iterations[2].setTime(2.f);
        ParticleSpecies& e_3 = o.iterations[3].particles["e"];
        e_3["position"][RecordComponent::SCALAR].resetDataset(dset);
        e_3["positionOffset"][RecordComponent::SCALAR].resetDataset(dset);
        o.iterations[3].setTime(3.f);
    }

Since no data is ever written, the ADIOS variable is not created in storage and not read back in.

Implementing a default as you suggest would be equivalent to writing dummy data, just at a later point.
If we go for this, we should keep #516 in mind. Should we?

@franzpoeschel franzpoeschel force-pushed the fix-empty-dataset-triggers-write branch from be566c8 to f6f75e7 Compare June 27, 2019 13:13
This test tested exactly the behaviour that this branch removes.
Also remove a number of duplicate tests.
@franzpoeschel franzpoeschel force-pushed the fix-empty-dataset-triggers-write branch from f6f75e7 to 2f67a6c Compare July 1, 2019 08:58
@ax3l ax3l merged commit 78001f3 into openPMD:dev Jul 1, 2019
@franzpoeschel franzpoeschel deleted the fix-empty-dataset-triggers-write branch January 28, 2021 13:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants