Memory benchmark trials (--OLD BROKEN VERSION : please see #5946 instead )

[pp-mo]

Superceded by #5946

WIP DO NOT MERGE THIS, EVER

Some modified code for memory benchmarks, evaluating "tracemalloc" against our existing Linux process-RSS measuring technique.

Additional context : sample output from a desktop run

$ asv run -b Memcheck* --quick -e -E existing:/tmp/persistent/newconda-envs/iris3/bin/python 
 . . .
· Discovering benchmarks
· Running 4 total benchmarks (1 commits * 1 environments * 4 benchmarks)
[ 0.00%] ·· Benchmarking existing-py_tmp_persistent_newconda-envs_iris3_bin_python
[ 0.00%] ··· Importing benchmark suite produced output:
[ 0.00%] ···· /net/home/h05/itpp/git/iris/iris_main/benchmarks/benchmarks/generate_data/__init__.py:52: UserWarning: No BENCHMARK_DATA env var, defaulting to /net/home/h05/itpp/git/iris/iris_main/benchmarks/.data. Note that some benchmark files are GB in size.
              warn(message)

[12.50%] ··· memtrace_evaluation.MemcheckBlocksAndWorkers.time_calc                                                                                                                                 ok
[12.50%] ··· ========= ========== ========== ==========
             --                    nworkers            
             --------- --------------------------------
              nblocks      1          4          9     
             ========= ========== ========== ==========
                 1      548±0ms    515±0ms    522±0ms  
                 4      119±0ms    165±0ms    175±0ms  
                 9      56.1±0ms   98.1±0ms   93.6±0ms 
             ========= ========== ========== ==========

[25.00%] ··· memtrace_evaluation.MemcheckBlocksAndWorkers.track_addmem_calc                                                                                                                         ok
[25.00%] ··· ========= ======= ======= =======
             --                nworkers       
             --------- -----------------------
              nblocks     1       4       9   
             ========= ======= ======= =======
                 1      305.3   305.3   305.3 
                 4       19.2    76.4    76.4 
                 9       4.0     15.2    33.9 
             ========= ======= ======= =======

[37.50%] ··· memtrace_evaluation.MemcheckRunstyles.time_calc                                                                                                                                        ok
[37.50%] ··· ============= ================= ================= =================== ===================
             --                                          runtype / ysize                              
             ------------- ---------------------------------------------------------------------------
                measure     threads / 10000   threads / 40000   processes / 10000   processes / 40000 
             ============= ================= ================= =================== ===================
              tracemalloc       96.7±0ms          267±0ms            48.7±0ms            87.2±0ms     
                  rss           97.4±0ms          258±0ms            47.6±0ms            92.7±0ms     
             ============= ================= ================= =================== ===================

[50.00%] ··· memtrace_evaluation.MemcheckRunstyles.track_addmem_calc                                                                                                                                ok
[50.00%] ··· ============= ================= ================= =================== ===================
             --                                          runtype / ysize                              
             ------------- ---------------------------------------------------------------------------
                measure     threads / 10000   threads / 40000   processes / 10000   processes / 40000 
             ============= ================= ================= =================== ===================
              tracemalloc         17.0              67.9               0.2                 0.2        
                  rss             9.1               34.3               0.8                 0.8        
             ============= ================= ================= =================== ===================

[pp-mo]

What I think we can deduce from the benchmark results

1. neither "rss" or "tracemalloc" tracks memory in subprocesses
   • compare MemcheckRunstyles.track_addmem_calc , any "threads" with "processes"
2. "rss" method is not picking up on memory used in multiple threads, but "tracemalloc" is
   • compare MemcheckRunstyles.track_addmem_calc , "threads" results ..
   • .. these are ~2 times as big for "tracemalloc" as for "rss" (this for nblocks=6, nworkers=4)