Issue #356 openadas interface by Didou09 · Pull Request #362 · ToFuProject/tofu

Didou09 · 2020-03-06T12:30:10Z

Motivations:

many diagnostics rely on the use of atomic data (for computing radiated power)
tofu does not provide atomic data, but it should provide an interface to download / use atomic data from several databases
One of the most easily accessible atomic databases is openadas, it is free and accessible online

Main changes:

an openadas2tofu sub-package is created
it provides routines for:
- exploring the online database from the ipyton console
- downloading selected files into the user's local ~/.tofu/ repo (created by tofu-custom, see PR User-customizable imas shortcuts #351, user-customizable scripts default parameters #359, Issue #359 Minor bug fix #361 )
- reading the downloaded files and returning the result as a python dict

Examples:

In [1]: import tofu as tf

# Search the online database for ionisation / recombination data (ADF11) for Argon
In [2]: tf.openadas2tofu.search_online('ar adf11', include_partial=False)                                                                                                                                                                                                          
Element  charge  type of data                               full file name           
-------  ------  -----------------------------------------  -------------------------
Ar               ECD: Mean ionisation potential             /adf11/ecd89/ecd89_ar.dat
Ar               PRC: Charge exchange emission              /adf11/prc89/prc89_ar.dat
Ar               ACD: Effective Recombination Coefficients  /adf11/acd74/acd74_ar.dat
Ar               SCD: Effective Ionisation Coefficients     /adf11/scd74/scd74_ar.dat
Ar               SCD: Effective Ionisation Coefficients     /adf11/scd85/scd85_ar.dat
Ar               PRB: Recombination and Bremsstrahlung      /adf11/prb89/prb89_ar.dat
Ar               PLT: Line Emission from excitation         /adf11/plt42/plt42_ar.dat
Ar               ACD: Effective Recombination Coefficients  /adf11/acd85/acd85_ar.dat
Ar               YCD: Mean charge state squared             /adf11/ycd89/ycd89_ar.dat
Ar               PLT: Line Emission from excitation         /adf11/plt89/plt89_ar.dat
Ar               ZCD: Mean charge state                     /adf11/zcd89/zcd89_ar.dat
Ar               PLS: Specific line emission                /adf11/pls89/pls89_ar.dat
Ar               CCD: CX Cross Coupling Coefficients        /adf11/ccd89/ccd89_ar.dat
Ar               SCD: Effective Ionisation Coefficients     /adf11/scd89/scd89_ar.dat
Ar               ACD: Effective Recombination Coefficients  /adf11/acd89/acd89_ar.dat
Ar               PLT: Line Emission from excitation         /adf11/plt40/plt40_ar.dat
Ar               PLT: Line Emission from excitation         /adf11/plt41/plt41_ar.dat

# Search the database by wavelength (in Angstrom) of spectral lines for ion Argon 16+
In [3]: tf.openadas2tofu.search_online_by_wavelengthA(lambmin=3.94, lambmax=4, element='ar')                                                                                                                                                                                       
Wavelength  Element  Charge  Data Type  Transition                               Full file name                          
----------  -------  ------  ---------  ---------------------------------------  ----------------------------------------
3.94817     Ar       16+     ADF15      11513 ^{1}P_{1.0} -> 21 ^{1}S_{0.0}      /adf15/pec40][ar/pec40][ar_ic][ar16.dat 
3.94817     Ar       16+     ADF15      11513 ^{1}P_{1.0} -> 21 ^{1}S_{0.0}      /adf15/pec40][ar/pec40][ar_ls][ar16.dat 
3.96042     Ar       16+     ADF15      11513 ^{0}S_{5.5} -> 21 ^{0}S_{0.0}      /adf15/pec40][ar/pec40][ar_ca][ar16.dat 
3.96042     Ar       16+     ADF15      11513 ^{0}S_{5.5} -> 21 ^{0}S_{0.0}      /adf15/pec40][ar/pec40][ar_cl][ar16.dat 
3.96445     Ar       16+     ADF15      11513 ^{3}P_{4.0} -> 21 ^{1}S_{0.0}      /adf15/pec40][ar/pec40][ar_ls][ar16.dat 
3.96626     Ar       16+     ADF15      11513 ^{3}P_{1.0} -> 21 ^{1}S_{0.0}      /adf15/pec40][ar/pec40][ar_ic][ar16.dat 
3.96731     Ar       15+     ADF15      11512513 ^{2}P_{1.5} -> 12 ^{2}S_{0.5}   /adf15/pec40][ar/pec40][ar_ic][ar15.dat 
3.96832     Ar       15+     ADF15      11512513 ^{2}P_{0.5} -> 12 ^{2}S_{0.5}   /adf15/pec40][ar/pec40][ar_ic][ar15.dat 
3.97233     Ar       15+     ADF15      11512513 ^{2}P_{2.5} -> 12 ^{2}S_{0.5}   /adf15/pec40][ar/pec40][ar_ls][ar15.dat 
3.97612     Ar       15+     ADF15      11512513 ^{2}P_{2.5} -> 12 ^{2}S_{0.5}   /adf15/pec40][ar/pec40][ar_ls][ar15.dat 
3.98011     Ar       15+     ADF15      11512513 ^{2}P_{1.5} -> 12 ^{2}S_{0.5}   /adf15/pec40][ar/pec40][ar_ic][ar15.dat 
3.98225     Ar       15+     ADF15      11512513 ^{2}P_{0.5} -> 12 ^{2}S_{0.5}   /adf15/pec40][ar/pec40][ar_ic][ar15.dat 
3.99122     Ar       15+     ADF15      11512513 ^{0}S_{11.5} -> 12 ^{0}S_{0.5}  /adf15/pec40][ar/pec40][ar_ca][ar15.dat 
3.99122     Ar       15+     ADF15      11512513 ^{0}S_{11.5} -> 12 ^{0}S_{0.5}  /adf15/pec40][ar/pec40][ar_cl][ar15.dat 
4           Ar       16+     ADF15       ^{3}S_{1.0} ->  ^{1}S_{0.0}             /adf15/transport/transport_llu][ar16.dat
4           Ar       16+     ADF15       ^{3}P_{1.0} ->  ^{1}S_{0.0}             /adf15/transport/transport_llu][ar16.dat

# download a specific file (and force update to overwrite if file already exists)
In [4]: tf.openadas2tofu.download('/adf15/pec40][ar/pec40][ar_ls][ar16.dat', update=True)                                                                                                                                                                                          
/Home/DV226270/ToFu_All/tofu_git/tofu/tofu/openadas2tofu/_requests.py:372: UserWarning: File already exists, will be downloaded and overwritten:
	/Home/DV226270/.tofu/openadas2tofu/adf15/pec40][ar/pec40][ar_ls][ar16.dat
  warnings.warn(msg)
file /adf15/pec40][ar/pec40][ar_ls][ar16.dat was copied to:
	/Home/DV226270/.tofu/openadas2tofu/adf15/pec40][ar/pec40][ar_ls][ar16.dat

# Read all downloaded adf15 files for Argon 16+ in desired wavelength range
In [5]: dout = tf.openadas2tofu.read_all(element='ar', typ1='adf15', charge=16, lambmin=3.94e-10, lambmax=4.e-10)                                                                                                                                                                  
	Loading data from /Home/DV226270/.tofu/openadas2tofu/adf15/pec40][ar/pec40][ar_ic][ar16.dat
	Loading data from /Home/DV226270/.tofu/openadas2tofu/adf15/pec40][ar/pec40][ar_cl][ar16.dat
	Loading data from /Home/DV226270/.tofu/openadas2tofu/adf15/pec40][ar/pec40][ar_ca][ar16.dat
	Loading data from /Home/DV226270/.tofu/openadas2tofu/adf15/pec40][ar/pec40][ar_ls][ar16.dat

Issues:

Fixes, in devel, issue #356

…ll and clean_downloads, TODO: add search_by_wavelength() and start routines for exploring local database

…ports

…lementing search_from_wavelength(), TBF

…d_all() all operational and consistent

…mes for ADF15, and lambmin, lambmax implemented for ADF15

…arge

…15 and adf11

pep8speaks · 2020-03-06T12:30:15Z

Hello @Didou09! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

In the file tofu/openadas2tofu/__init__.py:

Line 2:1: E265 block comment should start with '# '

Comment last updated at 2020-03-10 16:14:42 UTC

Didou09 · 2020-03-06T12:42:11Z

I can't get rid of the last PEP8 speak comment because that's the shebang (#! /usr/bin/python)

…read_adf15()

codecov-io · 2020-03-06T14:05:10Z

Codecov Report

Merging #362 into devel will decrease coverage by 0.66%.
The diff coverage is 8.44%.

@@            Coverage Diff             @@
##            devel     #362      +/-   ##
==========================================
- Coverage   39.24%   38.57%   -0.67%     
==========================================
  Files          85       88       +3     
  Lines       24625    25169     +544     
==========================================
+ Hits         9663     9709      +46     
- Misses      14962    15460     +498

Impacted Files	Coverage Δ
tofu/scripts/tofucustom.py	`0% <0%> (ø)`	⬆️
tofu/__init__.py	`100% <100%> (+6.06%)`	⬆️
tofu/version.py	`100% <100%> (ø)`	⬆️
tofu/openadas2tofu/_read_files.py	`5.66% <5.7%> (ø)`
tofu/openadas2tofu/__init__.py	`50% <50%> (ø)`
tofu/openadas2tofu/_requests.py	`8.42% <8.42%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 16f6c0b...3ee8536. Read the comment docs.

VEZINET Didier added 18 commits February 27, 2020 15:23

[Issue356] Created openadas2tofu sub-package with __init__.py

03b6311

[Issue356] updated version

cb0c53f

[Issue356] Advanced search()

7f7a339

[Issue356] Web interface operational for search, download, download_a…

687249c

…ll and clean_downloads, TODO: add search_by_wavelength() and start routines for exploring local database

[Issue356] Started reading routines, ADF15 to be finished and tested

d28fd3d

Merge branch 'devel' into Issue356_openadas

ecbc0a2

[Issue356] Debugged tofucustom for openadas/ creation and debugged im…

ee6f48b

…ports

[Issue356] ADF15 file reading ok with proper keys and checks, now imp…

a4c55fe

…lementing search_from_wavelength(), TBF

[Issue356] search_online(), search_online_by_wavelength() and downloa…

b7e9007

…d_all() all operational and consistent

[Issue356] read() operational for both absolute path and full adas na…

480d04d

…mes for ADF15, and lambmin, lambmax implemented for ADF15

[Issue356] read_all() operational, with element, Type and optional ch…

c42e00c

…arge

[Issue356] Debugged search_online() and now operational for adf11

39b1824

[Issue356] Replace '+' by '1+' in search_online()

11e72e0

[Issue356] Finishing ADF11/SCD reading routine

029850e

[Issue356] _read_adf11 operational and read_all() operational for adf…

f657de5

…15 and adf11

[Issue356] Harmonized output for adf11 and adf15 in read_all()

76d020d

[Issue356] Added docstr to read() and read_all()

f4208ff

[Issue356] Minor typo

26371bf

Didou09 added the New usage label Mar 6, 2020

Didou09 requested a review from lasofivec March 6, 2020 12:30

Didou09 self-assigned this Mar 6, 2020

VEZINET Didier added 2 commits March 6, 2020 13:38

[Issue356] PEPA compliance

2d4ab61

[Issue356] PEP8 compliance 2

328d320

Didou09 and others added 4 commits March 6, 2020 13:44

Merge branch 'devel' into Issue356_openadas

0d3af17

[Issue356] Typo in lambmin / lambmax selection in _read_adf15()

b9a55dc

[Issue356] Improve consistency with Issue202 for dlines dict out of _…

f832ab7

…read_adf15()

[Issue356] Typo

3ee8536

Merge branch 'devel' into Issue356_openadas

21beb70

lasofivec approved these changes Mar 10, 2020

View reviewed changes

lasofivec merged commit a7cffef into devel Mar 10, 2020

lasofivec deleted the Issue356_openadas branch March 10, 2020 16:20

Didou09 mentioned this pull request Mar 30, 2020

openadas interface #356

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Issue #356 openadas interface#362

Issue #356 openadas interface#362
lasofivec merged 25 commits intodevelfrom
Issue356_openadas

Didou09 commented Mar 6, 2020

Uh oh!

pep8speaks commented Mar 6, 2020 •

edited

Loading

Uh oh!

Didou09 commented Mar 6, 2020

Uh oh!

codecov-io commented Mar 6, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

Didou09 commented Mar 6, 2020

Motivations:

Main changes:

Examples:

Issues:

Uh oh!

pep8speaks commented Mar 6, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Comment last updated at 2020-03-10 16:14:42 UTC

Uh oh!

Didou09 commented Mar 6, 2020

Uh oh!

codecov-io commented Mar 6, 2020

Codecov Report

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

pep8speaks commented Mar 6, 2020 •

edited

Loading