lazy parsing of iterations

[franzpoeschel]

Opening a Series with many iterations is currently a time-intensive procedure due to eager parsing.
This adds an option to lazily parse a Series. If that option is chosen, Iteration::open() must be called before accessing. ReadIterations will do that automatically.

I previously attempted parsing iterations automatically upon accessing them via the Container interface, but there were too many edge cases and it's probably better to do such things explicitly in parallel situations.

TODO:

• It's probably time to use a builder pattern for the Series or something alike, the number of optional parameters keeps growing.
• Python bindings
• Documentation
• Cleanup, including removal of failed first attempt
• Support lazy parsing for group-based layout
• Merge Use schema-based versioning in ADIOS2 backend #941 first.
• This PR is affected by the issue described in Demonstrator PR to check whether our ICC tests are broken on dev #953. Worked around it in here by using auxiliary::Option instead of NaNs to represent no scaling.

[ax3l]

added inline :)

[ax3l]

Let's add a doxygen string here, just to clarify what this switch does.
Would DeferredReadAccess be more explicit? Not sure, ... maybe.

[franzpoeschel]

If we rename this one anyway, I'd go for DeferredParseAccess to avoid confusion with deferred load_chunks

[ax3l]

The implementation of the lazy/deferred parsing reads well. One general comment, I would unify the naming for this in the code base, at the moment we use:

• lazy
• deferred read
• deferred access
• (not) yet accessed

To describe the same thing, which can be confusing as the same concept in the code is meant.

Should we skip the introduction of the term "lazy" and instead use "deferred (iteration) access" in all places?

---

Do we want to make this a Series constructor parameter or JSON option?

---

Should we introduce the builder pattern in a separate PR and/or use it more consistently throughout the code base & examples?

[franzpoeschel]

The implementation of the lazy/deferred parsing reads well. One general comment, I would unify the naming for this in the code base, at the moment we use:

* lazy

* deferred read

* deferred access

* (not) yet accessed

To describe the same thing, which can be confusing as the same concept in the code is meant.

Should we skip the introduction of the term "lazy" and instead use "deferred (iteration) access" in all places?

I've renamed things now:

parse_lazily -> defer_iteration_parsing
NotYetAccessed -> ParseAccessDeferred
Iteration::deferRead -> Iteration::deferParseAccess
DeferredRead -> DeferredParseAccess
m_deferredRead -> m_deferredParseAccess
parseLazily -> runDeferredParseAccess

Do we want to make this a Series constructor parameter or JSON option?

Good idea, done

Should we introduce the builder pattern in a separate PR and/or use it more consistently throughout the code base & examples?

I introduced the builder pattern since it made for a better way to have a constructor with more than one defaulted argument. Since we're now back at one defaulted argument, we don't necessarily need it anymore. I've isolated the builder pattern to a branch franzpoeschel/topic-builder now, should I do a PR?

Also, we can definitely use it more in the examples and tests. In the code base itself probably not so much since we seldomly create a Series object ourselves.

[franzpoeschel]

If we rename this one anyway, I'd go for DeferredParseAccess to avoid confusion with deferred load_chunks

[ax3l]

Since you only call this in the .cpp file, you can define this helper function only in the Series.cpp file and in a namespace {} (as you already do) and avoid that it gets exposed as a function and symbol at all :)

[franzpoeschel]

It's not that easy because that function accesses private members. But we can theoretically make all members of SeriesData public and inherit it privately in SeriesInternal. So, the data struct can be passed to non-friend functions for manipulation?
Edit: Pushed a commit that does just that.
Edit: Not so easy either, this breaks dynamic casting. I've made it a public inheritance now, meaning that SeriesInternal has all members public now. Series still hides them.

[ax3l]

Clever :)

[ax3l]

Thank you, this is great!