Issue321 rectangular grid metis by Didou09 · Pull Request #325 · ToFuProject/tofu

Didou09 · 2020-01-10T13:21:41Z

Main changes:

Multi.to_Config() is now operational for vessel / limiter / mobile and also for limiter-only wall ids
Plasma2D now handles rectangular (R, Z) grids for interpolation, assuming the array order (shapeRZ=('R', 'Z') or ('Z', 'R') is provided by the user.
Multi.calc_signal() now properly handles METIS simulations with rectangular grids
Multi.to_Cam(), Multi.to_Data() and Multi.calc_signal() all now properly handle indch_auto=True (propagated)
Slightly more explicit warning in case of missing tofu sub-package

Some implementation details:

description_2d and indDescription were 2 different variable names for the same thing
=> description_2d is kept only
Multi._get_indch_geomtdata() hidden method implement to handle automated determination of indch in all cases (LOS, time and data), for readability and modularity
Shortcuts 2dmeshR and 2dmeshZ added for rectangular meshes

Status:

To this day (2020-01-10), on WEST intra servers, the following syntthetic diagnostics are operational directly from IMAS / METIS:

Polarimeter : ok
Interferometer : ok (provided METIS writes Nan outside of separatrix)
Bolometer: ok (provided Etendues are stored in ids and proper geometry)
Bremsstrahlung_visible: ok (provided proper units in ids and proper geometry)

The soft_x_rays ids is not operational yet because it requires atomic data, not implemented yet

Example (only on WEST intra servers):

In [1]: import tofu as tf

In [2]: multi = tf.imas2tofu.MultiIDSLoader(ids=['interferometer', 'polarimeter', 'spectrometer_visible', 'bolometer', 'bremsstrahlung_visible', 'soft_x_rays'], shot=54178, get=False)

In [3]: multi.add_ids_synthdiag(user='CB165101', shot=54178, run=13, tokamak='west')
Getting ids             [occ]  tokamak  user         version  shot   run  refshot  refrun
----------------------  -----  -------  -----------  -------  -----  ---  -------  ------
core_profiles           [0]    west     CB165101     3        54178  13   -1       -1    
core_sources            [0]    "        "            "        "      "    "        "     
equilibrium             [0]    "        "            "        "      "    "        "     
bolometer               [0]    west     imas_public  3        54178  0    -1       -1      
interferometer          [0]    "        "            "        "      "    "        "     
polarimeter             [0]    "        "            "        "      "    "        "     
pulse_schedule          [0]    "        "            "        "      "    "        "     
wall                    [0]    "        "            "        "      "    "        "     

In [4]: sig_pola = multi.calc_signal('polarimeter', indch_auto=True)
/Home/DV226270/ToFu_All/tofu_git/tofu/tofu/imas2tofu/_core.py:3165: UserWarning: indch set automatically for None
  (due to inhomogeneous data shapes)
    - main shape: (46446,)
    - nb. chan. selected: 8
    - indch: [2, 3, 4, 5, 6, 7, 8, 9]
  warnings.warn(msg)
/Home/DV226270/ToFu_All/tofu_git/tofu/tofu/imas2tofu/_core.py:2114: UserWarning: The following data could not be retrieved:
	- equilibrium:
		2dmeshFaces  : list index out of range
  warnings.warn(msg)

Issues:

Fixes, in devel, issue #321

…idMETIS

…for loading config from imas

…rategy : vessel in vessel, PFC in limiter or mobile units if any mobile, and more descriptions written

…mobile.unit, to avoid loss of info

…e and limiter

…ti.to_Cam() and Multi.to_Data(), hence 'names' field also added to all diag default shortcuts

…with METIS rectangular meshes

…ost done up to data

… now available for Multi.calc_signal()

pep8speaks · 2020-01-10T13:22:02Z

Hello @Didou09! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2020-01-14 10:05:00 UTC

codecov-io · 2020-01-10T14:45:38Z

Codecov Report

Merging #325 into devel will increase coverage by 0.02%.
The diff coverage is 0.97%.

@@            Coverage Diff             @@
##            devel     #325      +/-   ##
==========================================
+ Coverage   40.04%   40.06%   +0.02%     
==========================================
  Files          79       79              
  Lines       24108    24094      -14     
==========================================
  Hits         9653     9653              
+ Misses      14455    14441      -14

Impacted Files	Coverage Δ
tofu/data/_core.py	`38.97% <ø> (ø)`	⬆️
tofu/geom/_core.py	`63.19% <0%> (-0.09%)`	⬇️
tofu/imas2tofu/_core.py	`0.67% <0%> (ø)`	⬆️
tofu/data/_comp.py	`19.93% <0%> (-0.4%)`	⬇️
tofu/version.py	`100% <100%> (ø)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 762f2fe...74ab681. Read the comment docs.

Didou09 · 2020-01-10T14:56:52Z

-            vphi = -np.sin(phi)*vect[0,:] + np.cos(phi)*vect[1,:]
-            vZ = vect[2,:]
+            vR = np.cos(phi)*vect[0, :] + np.sin(phi)*vect[1, :]
+            vPhi = -np.sin(phi)*vect[0, :] + np.cos(phi)*vect[1, :]


vphi was incorrect, it is vPhi

Didou09 · 2020-01-10T14:57:47Z

                                indpts = indR*nZ + indZ
                            else:
                                indpts = indZ*nR + indR
+                            indout = ((r < R[0]) | (r > R[-1])


We make sure points outside of the rectangular grid return -1 as index

Didou09 · 2020-01-10T14:58:13Z

                              if self._dindref[id_]['group'] == 'mesh'][0]
-            pts = self.dmesh[idmesh]['data']['nodes']
-            pts = np.array([pts[:,0], np.zeros((pts.shape[0],)), pts[:,1]])
+            if self.dmesh[idmesh]['data']['type'] == 'rect':


Make sure the default pts are chosen according to the mesh type (rect vs triangular)

Didou09 · 2020-01-10T14:58:50Z

+        msg = "All Data objects do not have the same:\n"
        msg += "    dlabels['t'], dlabels['X'] and dlabels['data'] !"
-        raise Exception(msg)
+        warnings.warn(msg)


This is a non-critical error => downgraded from Exception to warning

Didou09 · 2020-01-10T14:59:24Z

        if Brightness is False:
            if dataname is None:
                dataname = r"LOS-integral x Etendue"
+            if E is None or np.all(np.isnan(E)):


Check etendue is correct before using it

Didou09 · 2020-01-10T15:02:14Z

-                if units[ii].phi_extensions.size > 0:
-                    pos, extent =  units[ii].phi_extensions.T
+            # Get vessel
+            nlim = len(wall.limiter.unit)


We now consider both limiter and mobile wall types

Didou09 · 2020-01-10T15:03:13Z

+            nmob = len(wall.mobile.unit)
+            onelimonly = False
+            try:
+                if len(wall.vessel.unit) != 1:


There should be only one vessel in the IMAS ids (corresponds to tofu call Ves : a StructIn sub-class)

Didou09 · 2020-01-10T15:04:28Z

+
+            # Determine if mobile or not
+            lS = []
+            if onelimonly is False:


If there are several PFC, we load them either from limiter or mobile

Didou09 · 2020-01-10T15:09:26Z


        return geomcls

+    def _get_indch_geomtdata(self, indch=None, indch_auto=None,


hidden method created to handle automated determination of indch in a modular way

Didou09 · 2020-01-10T15:13:49Z

-                node.phi_extensions = np.array([lS[ii].pos, lS[ii].extent]).T
-            node.closed = True
-            node.name = '%s_%s'%(lS[ii].__class__.__name__, lS[ii].Id.Name)
+        wall = idd.wall.description_2d[description_2d]


We make sure that we can save to vessel / limiter / mobile froma tofu Config

lasofivec · 2020-01-10T15:13:58Z


            # Interpolate
            indpts = trifind(r, z)
+            indok = indpts > -1


is this used after ?

Not yet, but it will be in a future version, just preparing for it

lasofivec · 2020-01-10T16:11:23Z

+            dcom = d0._extract_common_params(d0)
+
+        elif isinstance(other, np.ndarray):
            data = opfunc(d0.data, other)


I would put an "or" in the if, instead of showing two different cases with the same result

You(re right, but I wanted to separate these becasue I may to treat these cases differently later, let's say it for readability and anticipation of future needs

lasofivec · 2020-01-10T16:14:51Z

-            config = mod.Config(lStruct=lS, Name=Name, **kwargs)
+            if Name is None:
+                Name = wall.type.name
+                if Name == '':


is this rightly indented ?

yes, it happens that people save empty names ('') in IMAS, because they forget to fill it in

lasofivec · 2020-01-10T16:15:54Z

        shapeu = np.unique(np.r_[R.shape, Z.shape])
-        shapeRZ = [None, None]
+        if shapeRZ is None:
+            shapeRZ = [None, None]


should shapeRZ not be a tuple instead of a list (see below)

ultmately, shapeRZ will be returned as a tuple, yes,
But at this point we have to define it as a list because we may have to modify it, and tuples are immutable.
So we define it as a list, possibly modify it, and finally convert it to a tuple

…s nan already from source in Plasma2D.interp_pts2profile() Set sig = -2*sig for polarimeter due to vect direction (laser direction) Set vect = -u in newcalc=False Double checked all METIS synthetic diagnostics: polarimeter and interferometer ok bremsstrhalung_visible and bolometer waiting for sequenceur replay for units, geometry and etendues Synthetic diag now need to be double-checked for NICE / EQUINOX

…ctions. To be double-tested for all cases, METIS and NICE

…bile.units[-1]

…imiter

…n-zero and different in Multi.to_Config(), and limiter chosen by default (with warning) when they are equal

VEZINET Didier added 14 commits December 18, 2019 15:00

[Issue321] Added 2dmeshR and 2dmeshZ to default equilibirium shortcuts

1ce295f

Merge branch 'Issue202_SpectroX2DCrystal' into Issue321_RectangularGr…

bc47291

…idMETIS

[Issue321] Almost all synthdiag working, finish testing

8b7b95e

[Issue321] Warning more explicit for missing subpackages (print())

ccc8ff3

[Issue321] Harmonized variable name indDescription -> description_2d …

57ccad4

…for loading config from imas

[Issue321] imas2tofu._save_to_imas_Config() now complying with new st…

af693de

…rategy : vessel in vessel, PFC in limiter or mobile units if any mobile, and more descriptions written

[Issue321] info about whether Struct is mobile now stored in name of …

3940393

…mobile.unit, to avoid loss of info

Merge branch 'devel' into Issue321_RectangularGridMETIS

7b1cc0f

[Issue321] MultiIDSLoader.to_Config() now operational again from vell…

ac2a365

…e and limiter

[Issue321] Added 'names' to default data extracted when exporting Mul…

f7d47d5

…ti.to_Cam() and Multi.to_Data(), hence 'names' field also added to all diag default shortcuts

[Issue321] Interferometer and polarimeter synth diag now operational …

583cb9b

…with METIS rectangular meshes

[Issue321] Started refurbishing Multi.to_Data(indch, indch_auto), alm…

3ce09a5

…ost done up to data

Merge branch 'devel' into Issue321_RectangularGridMETIS

320dc7d

[Issue321] Synth. diag operational for all (tofu-side) and indch_auto…

75d20ea

… now available for Multi.calc_signal()

Didou09 added the imas interfacing - improve label Jan 10, 2020

Didou09 requested a review from lasofivec January 10, 2020 13:21

Didou09 self-assigned this Jan 10, 2020

VEZINET Didier added 8 commits January 10, 2020 14:29

[Issue321] PEP8 compliance

1b17fca

[Issue321] PEP8 compliance 2

456254d

[Issue321] PEP8 compliance 3

af676c7

[Issue321] PEP8 compliance 4

fac943f

[Issue321] PEP8 compliance 5

bb2a7e1

[Issue321] PEP8 compliance 6

5212640

[Issue321] PEP8 compliance 7

e74f678

[Issue321] PEP8 compliance 8

762f2fe

Didou09 commented Jan 10, 2020

View reviewed changes

lasofivec approved these changes Jan 10, 2020

View reviewed changes

Merge branch 'devel' into Issue321_RectangularGridMETIS

9fc4efe

VEZINET Didier added 6 commits January 13, 2020 17:11

[Issue321] Debugged time index bug in data._comp.py interpolation fun…

666ceff

…ctions. To be double-tested for all cases, METIS and NICE

[Issue321] Modified Config.save_to_imas() to put vessel in limiter/mo…

f22285a

…bile.units[-1]

[Issue321] Modified Multi.to_Config() to handle ves as last unit of l…

b571227

…imiter

[Issue321] More explicit error message when nlim and nmob are both no…

5e9b601

…n-zero and different in Multi.to_Config(), and limiter chosen by default (with warning) when they are equal

[Issue321] PEP8 compliance 9

74ab681

Didou09 merged commit 80bc6fe into devel Jan 14, 2020

Didou09 deleted the Issue321_RectangularGridMETIS branch January 14, 2020 12:25

Didou09 mentioned this pull request Jan 30, 2020

tofu 1.4.2 preparation #339

Merged

Didou09 mentioned this pull request Mar 10, 2020

Preparing 1.4.3 #364

Merged


		return geomcls

		def _get_indch_geomtdata(self, indch=None, indch_auto=None,

Conversation

Didou09 commented Jan 10, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Main changes:

Some implementation details:

Status:

Example (only on WEST intra servers):

Issues:

Uh oh!

pep8speaks commented Jan 10, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Comment last updated at 2020-01-14 10:05:00 UTC

Uh oh!

codecov-io commented Jan 10, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Didou09 commented Jan 10, 2020 •

edited

Loading

pep8speaks commented Jan 10, 2020 •

edited

Loading

codecov-io commented Jan 10, 2020 •

edited

Loading