Skip to content

Remove dummy dataset writing from PatchRecordComponent::flush()#512

Merged
ax3l merged 1 commit intoopenPMD:devfrom
franzpoeschel:fix-empty-dataset-triggers-write
May 16, 2019
Merged

Remove dummy dataset writing from PatchRecordComponent::flush()#512
ax3l merged 1 commit intoopenPMD:devfrom
franzpoeschel:fix-empty-dataset-triggers-write

Conversation

@franzpoeschel
Copy link
Contributor

Proposal to fix #489, see there for the underlying issue.
As described there, the current state causes issues in other backends. Notably, ADIOS2 will not reliably delete the dummy value upon writing the actual data, so the output may contain random data at the first position.
Proposed solution in this PR:

  • Remove this workaround from the frontend. It addresses a backend-specific issue and should be resolved in the backend, not the frontend. The above issue supports this point.
  • For now, do not implement an alternative solution to the issue addressed by the workaround, i.e. ADIOS variables only being physically created after writing data. Up to discussion.
    • Option 1: Keep things this way. It is a design decision in ADIOS (also ADIOS2) not to physically create variables (used for openPMD datasets) unless data is actually written and openPMD can leverage this behavior. The workaround removed in this PR tries to circumvent this behavior. Needs to be documented in this case.
    • Option 2: Implement an alternative workaround. I see no clean way to do this at the moment.

Todo:

  • Document the changing behavior, especially in case we go with option 1.

Due to the issues described above, this should be merged before #482.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Flush Before storeChunk: Triggers Write

2 participants