MPI Benchmark by franzpoeschel · Pull Request #346 · openPMD/openPMD-api

franzpoeschel · 2018-09-14T14:41:41Z

Adds a generic MPI benchmark to openPMD (#37).

Todo:

The reading benchmark fails when using the ADIOS1 backend (openPMD::unsupported_data_error: Unsupported attribute datatype)
Feature requests?
Resolve strange MPI behaviour (see MPIBenchmarkReport.hpp → addReport() method)
Adjust documentation to refactoring
Add user documentation (*.rst).
Add licensing information

ax3l · 2018-09-25T08:38:53Z

Just as a general feature request: it would be cool if this tool allows easy, user-level performance benchmarks on clusters similar to what we did in Figure 1 in https://arxiv.org/abs/1706.00522 . So that we can easily switch backends, compression, written size per node and the like.

franzpoeschel · 2018-09-26T13:42:13Z

Thanks for your suggestions, the figure gives a good guideline.
I am not yet fully done (currently having a little fight with MPI), but next to some refactoring I have added support for picking multiple backends and multiple numbers of threads to be used for the benchmark.
The total extent of the written dataset is still fixed during one benchmark to give comparable results. Do you think it would be useful being able to instruct the Benchmark to be executed for several dataset extents?
So in total, the user currently picks

compression schemes
backends (per filename extension)
number of threads to be used

and the benchmark is then executed for every possible combination of entered values.

↑

edit: this has changed now, look into the documentation

ax3l · 2018-10-03T16:25:48Z

Do you think it would be useful being able to instruct the Benchmark to be executed for several dataset extents?

Yep, sounds useful since it controls the data per node.

We should write both particle and mesh data, but I am ok to start with the latter.

franzpoeschel · 2018-10-07T14:08:50Z

Yep, sounds useful since it controls the data per node.

I have expressed myself unclearly, the Benchmark class already allows choosing the part of the dataset to write per rank relatively freely. The user can choose this be passing to the constructor an implementation of the BlockSlicer abstract class. Once the BlockSlicer is chosen, it is currently fixed to allow for comparable results – unlike compression strategies, backends and thread sizes which can be altered.

We should write both particle and mesh data, but I am ok to start with the latter.

To be fair, I just copied that part from here, I will look into it.

ax3l · 2018-11-05T07:28:01Z

docs/source/index.rst

+   :maxdepth: 1
+   :hidden:
+
+   /utilities/benchmark.rst


does not need an initial /

ax3l · 2018-11-05T07:28:54Z

src/Datatype.cpp

+        os << dt;
+        return buf.str();
+    }
+}


missing single newline before EOF

ax3l · 2018-11-05T07:29:11Z

src/benchmark/mpi/OneDimensionalBlockSlicer.cpp

+
+#endif
+}
+


one newline too much ;)

ax3l · 2018-11-05T07:29:26Z

include/openPMD/benchmark/mpi/RandomDatasetFiller.hpp

+
+    };
+
+}


missing newline before EOF

ax3l · 2018-11-05T07:29:38Z

include/openPMD/benchmark/mpi/OneDimensionalBlockSlicer.hpp

+    };
+}
+#endif
+


one newline too much ;)

ax3l · 2018-11-05T07:29:46Z

include/openPMD/benchmark/mpi/MPIBenchmarkReport.hpp

+}
+
+#endif
+


one newline too much ;)

ax3l · 2018-11-05T07:39:13Z

CMakeLists.txt

        src/backend/PatchRecordComponent.cpp
-        src/backend/Writable.cpp)
+        src/backend/Writable.cpp
+        src/benchmark/mpi/OneDimensionalBlockSlicer.cpp)


Is it possible that this file and mpi_benchmark.cpp should be just in an extra or example target list?

I added mpi_benchmark.cpp (now 8_mpi_benchmark.cpp) to the example target list.
For OneDimensionalBlockSlicer.cpp do you mean to create a new list for that?

Yes, this .cpp file is not part of the library's CORE_SOURCE but of the examples/apps only.

docs/source/utilities/mpi_benchmark.cpp

+    }
+
+    MPI_Finalize();
+}


Implementation for slicing a hypercube along one dimension Refactor MPI benchmarking Dataset filling is now implemented by an own class instead of a virtual function. Add the fundaments for a write benchmark Adapted from https://openpmd-api.readthedocs.io/en/latest/usage/parallel.html#writing Nearly complete, but untested MPI benchmark Missing the collection of benchmarking results for the single threads Basically executable version of the benchmark Better handling of dataset indexing Do not set a compression when the compression string is empty Fix index calculation Create the write data anew for every iteration This allows for bigger benchmarks since only final memory is needed, but the time for data creation needs to be measured and deducted in the end. Also document the code. Refactor the creation of a DataSet Forward constructor arguments to the DatasetFiller Add some documentation and cleanup Flush the Series after every iteration in the benchmark Integrate with the build system Allow compiling without MPI enabled Also fix some shadowed variables Refactor the benchmark Make BlockSlicer and DatasetFiller abstract classes instead of templates. Add configuration options for backends and numbers of threads to use. Make it possible to actually include the benchmark. Refactor benchmark to allow for several datatypes in one run Make the benchmark system more flexible Includes setting the number of iterations per benchmark configuration and adding an adaptor class template SimpleDatasetFillerProvider lifting a DatasetFiller to a DatasetFillerProvider. Add documentation for the benchmark Includes an example usage.

"resolve MPI misbehaviour" = "realized that I had a faulty installation"

Add licensing information. Fix whitespaces. Add the MPI Benchmark example from the documentations to the examples to be built during openMPI building.

ax3l · 2018-11-23T10:25:21Z

include/openPMD/Datatype.hpp

+ * openPMD type.
+ */
+
+#if _MSC_VER && !__INTEL_COMPILER


Please place this macro block above the doxygen block (separated by an empty newline)

ax3l · 2018-11-30T10:50:39Z

I can merge this for now, thanks a lot!

ax3l · 2018-12-13T10:57:47Z

@franzpoeschel in CI, we get sporadically the error

what():  Internal error: Failed to create directories during ADIOS file creation

in the MPI benchmark. Can you confirm and might there be a quirk we need to fix?

franzpoeschel · 2018-12-13T11:08:39Z

I have seen this error already, yes. I thought it was an issue with the ADIOS backend, I will try finding the cause of it.

franzpoeschel · 2019-01-09T09:35:47Z

Just to be sure, #418 is exactly this issue, right?

ax3l · 2019-01-09T11:03:42Z

yes, #418 and #421 were sporadic CI issues with parallel I/O :)

franzpoeschel force-pushed the topic-mpiBenchmark branch from 133bb9a to 3b86b64 Compare September 14, 2018 16:31

ax3l assigned ax3l and anokfireball Sep 15, 2018

ax3l requested review from anokfireball and ax3l September 15, 2018 00:41

ax3l added MPI tests labels Sep 15, 2018

franzpoeschel force-pushed the topic-mpiBenchmark branch from 3b86b64 to 79a2a01 Compare September 15, 2018 07:55

franzpoeschel force-pushed the topic-mpiBenchmark branch from 72aeec7 to 698edde Compare September 25, 2018 09:04

franzpoeschel force-pushed the topic-mpiBenchmark branch from 6854b1b to 3160119 Compare September 29, 2018 19:48

ax3l mentioned this pull request Oct 4, 2018

RODARE: Contributors #378

Merged

franzpoeschel force-pushed the topic-mpiBenchmark branch from 3160119 to 0022fbb Compare October 7, 2018 13:56

franzpoeschel force-pushed the topic-mpiBenchmark branch 2 times, most recently from 1ba8b5d to 53129c6 Compare October 7, 2018 14:42

franzpoeschel force-pushed the topic-mpiBenchmark branch 3 times, most recently from 4ffe131 to b3525a1 Compare October 29, 2018 16:20

franzpoeschel force-pushed the topic-mpiBenchmark branch from 62a848b to 7a3a838 Compare November 4, 2018 19:59

ax3l reviewed Nov 5, 2018

View reviewed changes

franzpoeschel force-pushed the topic-mpiBenchmark branch 4 times, most recently from c54507c to 58315a1 Compare November 7, 2018 14:06

franzpoeschel added 2 commits November 14, 2018 10:42

Fix error, resolve MPI misbehaviour

2a570a7

"resolve MPI misbehaviour" = "realized that I had a faulty installation"

franzpoeschel force-pushed the topic-mpiBenchmark branch from 58315a1 to b8751b6 Compare November 14, 2018 09:48

franzpoeschel force-pushed the topic-mpiBenchmark branch 3 times, most recently from 0354bec to ed1fa9c Compare November 14, 2018 14:50

Cleanup

46b18f2

Add licensing information. Fix whitespaces. Add the MPI Benchmark example from the documentations to the examples to be built during openMPI building.

franzpoeschel force-pushed the topic-mpiBenchmark branch from ed1fa9c to 46b18f2 Compare November 14, 2018 15:08

Document the example usage better

6b729e9

ax3l reviewed Nov 23, 2018

View reviewed changes

ax3l approved these changes Nov 30, 2018

View reviewed changes

ax3l merged commit 7f6b29c into openPMD:dev Nov 30, 2018

ax3l mentioned this pull request Nov 30, 2018

Parallel Benchmark: Cleanup #398

Merged

ax3l mentioned this pull request Dec 13, 2018

Example: MPI Benchmark #37

Closed

franzpoeschel deleted the topic-mpiBenchmark branch January 28, 2021 13:28


		#endif
		}

		}

		#endif

Conversation

franzpoeschel commented Sep 14, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ax3l commented Sep 25, 2018

Uh oh!

franzpoeschel commented Sep 26, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ax3l commented Oct 3, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

franzpoeschel commented Oct 7, 2018

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ax3l Nov 8, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This comment was marked as resolved.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ax3l commented Nov 30, 2018

Uh oh!

ax3l commented Dec 13, 2018

Uh oh!

franzpoeschel commented Dec 13, 2018

Uh oh!

franzpoeschel commented Jan 9, 2019

Uh oh!

ax3l commented Jan 9, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

franzpoeschel commented Sep 14, 2018 •

edited

Loading

franzpoeschel commented Sep 26, 2018 •

edited

Loading

ax3l commented Oct 3, 2018 •

edited

Loading

ax3l Nov 8, 2018 •

edited

Loading