Read backend configuration from .toml files

[franzpoeschel]

Approach for implementation: Read from .toml file, but convert to nlohmann::json internally

Config files are much more readable in TOML:

unused = "global parameter"

[hdf5]
unused = "hdf5 parameter please dont warn"

[adios2]
unused = "parameter"

[adios2.engine]
type = "bp3"
unused = "as well"

[adios2.engine.parameters]
BufferGrowthFactor = 2
Profile = "On"

# double brackets, because the operators are a list
[[adios2.dataset.operators]]
type = "blosc"

# parameters for the current list entry
[adios2.dataset.operators.parameters]
parameters.clevel = "1"
parameters.doshuffle = "BLOSC_BITSHUFFLE"

TODO:

• parallel testing
• merge JSON update #1043 and Thirdparty library: toml11 by Toru Niina #1148 first
• error messages don't yet distinguish between JSON and TOML
• make TOML usable for Dataset config too (how to do that?)
• Make TOML an optional dependency? discussed: compiler coverage here is good.
• Separate toml11 thirdparty lib addition into separate PR Thirdparty library: toml11 by Toru Niina #1148
• Documentation thirdparty lib: as with nlohmann::json (news file, readme, ...) Thirdparty library: toml11 by Toru Niina #1148
• Update CommonADIOS1IOHandlerImpl< ChildClass >::initJson() after merging Fix reading of dataset-specific configuration in ADIOS1 #1162 (there will be no merge conflict)
• Add a CI config that tests this again upstream toml11 -> covered in nightly install tests

[franzpoeschel]

When I leave out the filehandle.good() check on Windows, I get this error:

C:\Users\Franz\git-repos\openPMD-api\test\SerialIOTest.cpp(3352): FAILED:
due to unexpected exception with message:
string too long 

I'll have to try using a standalone toml11 example to see if there is a bug on Windows.

[ax3l]

Rebased after #1148 was merged :)

[franzpoeschel]

Rebased after #1148 was merged :)

We'll have to watch out when rebasing as soon as #1043 is merged, since this rebase duplicated the commits of that PR

[franzpoeschel]

Rebased and up for review :) @ax3l

[ax3l]

Thank you, the code looks great to me! ✨

In order to finalize the PR for merge, can you please:

• add in the manual according sections (the same way as we document user-facing JSON options right now)
• add a user-facing example case

? :)

[franzpoeschel]

Thank you, the code looks great to me! sparkles

In order to finalize the PR for merge, can you please:

* add in the manual according sections (the same way as we document user-facing JSON options right now)

* add a user-facing example case

? :)

Still in jetlag? :D

Yep, those are good suggestions. I'll try to work the TOML configuration into the docs (since both are equivalent, I think I'll not duplicate everything, but have some examples for TOML here and there), also I think for user-facing examples you mean something under the /examples folder? If yes, yep I can add something there.

[franzpoeschel]

I've now added documentation and a new example based on the streaming examples.

[lgtm-com]

This pull request introduces 1 alert when merging 94364d9 into 05eff02 - view on LGTM.com

new alerts:

• 1 for Unused import

[ax3l]

Looks great, minor final updates inline :)

Also a small collision from #1162 in src/IO/ADIOS/CommonADIOS1IOHandler.cpp now.

[franzpoeschel]

Also a small collision from #1162 in src/IO/ADIOS/CommonADIOS1IOHandler.cpp now.

rebased