Pickle API: Avoid static Series hack, allow multiple Series

[franzpoeschel]

Close #1458

Theoretically, it should be possible to modify our public API such that each handle, not only the Series handle would keep alive the Series, but that would require some greater internal restructuring.

Instead, this PR avoids the problem of finding a place to store the Series not by putting it in a static variable, but by hiding it inside the returned value's destructor. This does not lead to a reference cycle because the returned handle is a non-owning shared pointer whose destructor lambda owns all actual data. As soon as the handle is let go, the destructor is run and the owned objects are freed.

• We could still use static (or better thread_local) to cache the latest opened Series for workflows that unpickle lots of single objects
• Maybe take the chance to introduce pickling for Series and Iteration

@pordyna Can you check if this helps your use case?

[pordyna]

@franzpoeschel Just tested and it works. I can now change series within one kernel instance.
Btw. I didn't new that, but one can install the api from a specific branch with pip like this pip install git+https://github.com/franzpoeschel/openPMD-api.git@unpickle-multiple-series, pretty cool.

[franzpoeschel]

Nice, thank you for testing this!

Btw. I didn't new that, but one can install the api from a specific branch with pip like this pip install git+https://github.com/franzpoeschel/openPMD-api.git@unpickle-multiple-series, pretty cool.

I didn't know that either. But I guess it means that you need to install ADIOS2 or HDF5 manually before that since they have no Pip packages? We usually just build them into the Wheel, but that doesn't help when compiling manually.

[pordyna]

I didn't know that either. But I guess it means that you need to install ADIOS2 or HDF5 manually before that since they have no Pip packages? We usually just build them into the Wheel, but that doesn't help when compiling manually.

I have installed adios2, hdf5, and blosc with conda. But see, https://pypi.org/project/adios2/ there is adios on pypi.

[franzpoeschel]

I have installed adios2, hdf5, and blosc with conda. But see, https://pypi.org/project/adios2/ there is adios on pypi.

Ah right, they actually have a package by now. Maybe we should add this as an optional dependency in our Pip-package @ax3l

[ax3l]

Let's add a nice doxygen string here :D

[franzpoeschel]

maybe document the idea here with a little inline comment

[franzpoeschel]

let's isolate this to a separate PR which goes into the 0.17 development branch