Issue346 tofucalc mat

[Didou09]

Motivations:

• synthetic diagnostics computation is easily done if the user compares experimental data to synthetic data calculated from the same ids, or from a different ids
• But, for the sake of speed, some users want to be able to easily calculate syntehtic data not from an ids but from a mat file if which an ids has been saved
• This use case is, at the moment very specific to the data treatment chain of one diagnostic (bremsstrhalung_visible) on WEST, so an ad hoc solution is proposed
• We will see on the longer term if it remains an exception or if becomes interesting to generalize this use-case or not

Main changes:

• A new module _mat2ids2calc.py has been created in sub-package tofu/imas2tofu/ to handle reading from the mat file, asusming its structure mimics the ids _core_profiles_ (all other data is taken from the main ids: diag geometry, equilibrium...)
• New keyword arguments input_file and output_file have been adde to generic function tofu.utils.calc_from_imas() so the user can specify the file for input and, optionally, a mat file where the synthetic signal will be saved.
• These keyword arguments have been ported to the bash script interface tofucalc.py for easy use from matlab or bash
• A few other minor bug fixes / minor improvements, inc. in particular the default shortcuts of MultiIDSLOader moved to a separate file tofu/imas2tofu/_def.py and updates to the shortcuts themselves
• tofu/data/_physics.py moved one step up to tofu/_physics.py

Example (IRFM intra only, on the proper git branch):

In the example below, tofu loads 4 ids from IMAS (the diagnostic to get the geometry, the equilibrium for 2D interpolation, the wall for the geometry and pulse_schedule which is always loaded by default).
It loads the 1d core profiles directly from the provided input_file
After a summary and some harmless warnings, we get a message confirming the synthetic data was successfully saved in the desired .mat file.

(base) [ DV226270 spica ~/ToFu_All/tofu_git/tofu (Issue346_tofucalcMat)] ./tofu/scripts/tofucalc.py -s 55983 -i bremsstrahlung_visible --input_file inputs_temp/one_brem.mat --output_file tests.mat
/Home/DV226270/ToFu_All/tofu_git/tofu/tofu/__init__.py:95: UserWarning: 
The following subpackages are not available:
    - tofu.mag
  => see print(tofu.dsub[<subpackage>]) for details.
  warnings.warn(msg)
/Home/DV226270/ToFu_All/tofu_git/tofu/tofu/__init__.py 1.4.2b4-44-g919e2cba
Getting ids             [occ]  tokamak  user         version  shot   run  refshot  refrun
----------------------  -----  -------  -----------  -------  -----  ---  -------  ------
bremsstrahlung_visible  [0]    west     imas_public  3        55983  0    -1       -1    
equilibrium             [0]    "        "            "        "      "    "        "     
pulse_schedule          [0]    "        "            "        "      "    "        "     
wall                    [0]    "        "            "        "      "    "        "     
/Home/DV226270/ToFu_All/tofu_git/tofu/tofu/imas2tofu/_core.py:1760: RuntimeWarning: invalid value encountered in greater
  out[ii][np.abs(out[ii]) > 1.e30] = np.nan
/Home/DV226270/ToFu_All/tofu_git/tofu/tofu/imas2tofu/_core.py:1934: UserWarning: The following data could not be retrieved:
	- equilibrium:
		2dmeshR  : list index out of range
		2dmeshZ  : list index out of range
		x1  : 'x1R'
		x1R  : list index out of range
		x1Z  : list index out of range
  warnings.warn(msg)
/Home/DV226270/ToFu_All/tofu_git/tofu/tofu/imas2tofu/_core.py:2178: UserWarning: 
ids wall has more limiter than mobile units
	- len(wall.description_2[1].limiter.unit) = 13
	- len(wall.description_2[1].mobile.unit) = 1
  => Choosing limiter by default
  warnings.warn(msg)
/Applications/Anaconda/python37/lib/python3.7/site-packages/scipy/interpolate/interpolate.py:687: RuntimeWarning: invalid value encountered in less
  below_bounds = x_new < self.x[0]
/Applications/Anaconda/python37/lib/python3.7/site-packages/scipy/interpolate/interpolate.py:688: RuntimeWarning: invalid value encountered in greater
  above_bounds = x_new > self.x[-1]
Successfully saved in:
	tests.mat

Pro tip: if --output_file is not provided, tofu plots the result in an interactive figure for visualization

(base) [ DV226270 spica ~/ToFu_All/tofu_git/tofu (Issue346_tofucalcMat)] ./tofu/scripts/tofucalc.py -s 55983 -i bremsstrahlung_visible --input_file inputs_temp/one_brem.mat

[pep8speaks]

Hello @Didou09! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2020-02-20 16:09:00 UTC

[codecov-io]

Codecov Report

Merging #352 into devel will decrease coverage by 0.31%.
The diff coverage is 4.52%.

@@            Coverage Diff             @@
##            devel     #352      +/-   ##
==========================================
- Coverage   39.64%   39.32%   -0.32%     
==========================================
  Files          81       83       +2     
  Lines       24363    24536     +173     
==========================================
- Hits         9659     9650       -9     
- Misses      14704    14886     +182

Impacted Files	Coverage Δ
tofu/_physics.py	25% <ø> (ø)
tofu/utils.py	47.15% <0%> (-0.96%)	⬇️
tofu/imas2tofu/_mat2ids2calc.py	0% <0%> (ø)
tofu/imas2tofu/__init__.py	30.5% <0%> (-1.08%)	⬇️
tofu/scripts/tofucalc.py	0% <0%> (ø)	⬆️
tofu/version.py	100% <100%> (ø)	⬆️
tofu/imas2tofu/_def.py	100% <100%> (ø)
tofu/imas2tofu/_core.py	0.8% <23.52%> (+0.15%)	⬆️
tofu/data/_core_new.py	13.15% <50%> (-0.47%)	⬇️
tofu/data/_core.py	38.79% <50%> (-0.18%)	⬇️
... and 6 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update dd17eb2...cd441ad. Read the comment docs.

[lasofivec]

Thank you for the detailed message !
A quick question looks like there is a NaN during the computation, I imagine this is normal ?

/Home/DV226270/ToFu_All/tofu_git/tofu/tofu/imas2tofu/_core.py:1760: RuntimeWarning: invalid value encountered in greater
out[ii][np.abs(out[ii]) > 1.e30] = np.nan

PS: I didn't deleted the branch

[Didou09]

Yep, it happens that there are NaNs, this is from IMAS data, so it depends on what is stored in IMAS
Thanks !