Add KIT phantom dataset #12105

JD-Zhu · 2023-10-12T07:48:03Z

I've uploaded the data to OSF but not sure how to proceed. Would appreciate any help!

What does this implement/fix?

Add new dataset - KIT phantom data.

Add KIT phantom data

mscheltienne · 2023-10-12T10:07:03Z

Thank you, it looks like a good start, you'll also need to add the dataset information (URL, hash) to the configuration file https://github.com/mne-tools/mne-python/blob/main/mne/datasets/config.py and to the documentation build https://github.com/mne-tools/mne-python/blob/main/doc/api/datasets.rst

larsoner · 2023-10-12T15:23:30Z

Looking back at a recent dataset addition d757026 I've edited your top comment to add checkboxes for the files that you'll want to make equivalent changes in (including config.py that @mscheltienne mentioned). There are a lot but looking at what's there hopefully it's easy enough to follow the patterns.

If you'd like help with this part don't hesitate to ask -- I can push a quick commit to do everything except adding the new tutorials/inverse/95_phantom_kit.py. Hopefully you can create that, look at existing tutorials/inverse/*phantom*.py files for examples!

JD-Zhu · 2023-10-16T07:02:50Z

Looking back at a recent dataset addition d757026 I've edited your top comment to add checkboxes for the files that you'll want to make equivalent changes in (including config.py that @mscheltienne mentioned). There are a lot but looking at what's there hopefully it's easy enough to follow the patterns.

If you'd like help with this part don't hesitate to ask -- I can push a quick commit to do everything except adding the new tutorials/inverse/95_phantom_kit.py. Hopefully you can create that, look at existing tutorials/inverse/*phantom*.py files for examples!

Thanks a lot for the help! I've updated most of the files, except for "doc/overview/datasets_index.rst" which I couldn't locate.

Happy to try creating the tutorial - but I may need to come back and do this a bit later!

larsoner · 2023-10-16T14:20:21Z

Looks like that file has moved to doc/documentation/datasets.rst

JD-Zhu · 2023-10-17T02:38:18Z

Thanks - now updated.

Some of the checks are failing and I'm not sure why - could you help me with these?

tutorials/inverse/95_phantom_KIT.py

JD-Zhu · 2023-10-21T02:39:03Z

Thank you @larsoner !

I have fixed the login for CircleCI - please let me know if this still causes issues.

I'm still happy to attempt the analysis, but it might just take a while as I have quite a lot going on at the moment. To test things locally, how do I get it to run the version of code I'm working on rather than the mne version already installed on my computer?

Alternatively, if you are happy to do the analysis, that'd also be great! I think there are 49 dipoles activated in sequence within each "train", with each activation being a sinusoidal signal at 11Hz (see MISC 017). Please let me know if you require more information.

Do we need to get the "ground truth" dipole locations in order to verify the accuracy of source localisation?

larsoner · 2023-10-21T13:02:35Z

It would improve the tutorial to know the true locations, yeah

larsoner · 2023-10-23T14:59:04Z

tutorials/inverse/95_phantom_KIT.py

+# We need to correct the ``dev_head_t`` because it's incorrect for these data!
+# relative to the helmet: hleft, forward, up
+translation = mne.transforms.translation(x=0.008, y=-0.025, z=-0.092)
+# pitch down (rot about x/R), roll left (rot about y/A), yaw left (rot about z/S)
+rotation = mne.transforms.rotation(
+    x=np.deg2rad(5),
+    y=np.deg2rad(-0.5),
+    z=np.deg2rad(-3),
+)


@JD-Zhu the info['dev_head_t'] / MEG-to-head transformation for this dataset starts out as an identity matrix. This is incorrect, so I figured out some rotation and translation params by eye under the assumption that the phantom was symmetrical (which is probably a good assumption but by-eye adjustment is not great). Should we in principle be able to get a correct dev_head_t from this dataset, or was it not measured during acquisition? If it was measured, it suggests we need to add or fix something in MNE so that dev_head_t is not identity on read / analysis...

I'm not too sure how to get the correct matrix - does this have to do with the marker coils and head shape digitisation? For KIT data, these are recorded in separate files (rather than being embedded in the raw MEG data). For this phantom dataset, we did marker coil measurements (_ini.mrk) but not head digitisation.

Ahh bummer. Yes I think you need both. If you still have the phantom and know where you had the coils you could maybe redigitize then update the dataset. But that sounds like a lot of work :)

We no longer have the phantom unfortunately, but the marker coils were attached at fixed locations (i.e. there were designated pins on the phantom), so I think it may be possible to get the coordinates for these somehow.

Okay I think we can punt on this for now, then. We can always add these to the dataset at a later date if you track down the phantom and can do a marker measurement. Would be cool to see how close it is to my hand-optimized values. 🙂

Haha yes, would be interesting to see!

larsoner · 2023-10-23T15:01:40Z

mne/event.py

+                f"Trigger channel {ch_name} has a non-zero initial value of "
+                "{initial_value} (consider using initial_event=True to detect this "


While analyzing this dataset I initially tried mne.find_events on it and it wasn't immediately obvious to me what stim channel was actually being used, so I improved mne.find_events to log what it's actually using as the event channel.

larsoner · 2023-10-23T15:02:09Z

mne/transforms.py

-            [0, 0, 0, 1],
-        ],
-        dtype=float,
-    )


I noticed this was identical to the code below so I DRY'ed it

larsoner · 2023-10-23T16:44:57Z

@JD-Zhu based on the symmetry I'm assuming this is correct:

https://output.circle-artifacts.com/output/job/4233b1c2-26a5-49f0-aecc-d996027fd63e/artifacts/0/html/auto_tutorials/inverse/95_phantom_KIT.html#sphx-glr-auto-tutorials-inverse-95-phantom-kit-py

But it would be good to get the dipole locations if possible. And we should really fix the dev_head_t if possible as well (see comment above).

JD-Zhu · 2023-10-27T06:35:10Z

This is awesome! Thanks @larsoner

The dipole locations are available in this paper (see Table 3):
https://pubmed.ncbi.nlm.nih.gov/25985908/

larsoner · 2023-10-27T14:49:40Z

The dipole locations are available in this paper (see Table 3):

Awesome, I'll look soon!

larsoner · 2023-10-27T14:49:54Z

Sorry, errant close!

* upstream/main: (26 commits) FIX: Fix bug with coreg scalars (mne-tools#12164) Changed casting rule in np.clip to allow reading of raw GDF files (mne-tools#12168) [DOC] Add documentation for setting montage order (mne-tools#12160) Fix inferring fiducials from EEGLAB (mne-tools#12165) Try to fix ICA Report (mne-tools#12167) BUG: Fix bug with Report.add_ica component number (mne-tools#12156) MAINT: Add rstcheck to CIs and pre-commit (mne-tools#12163) DOC: fix sphinx style typos (mne-tools#12161) MAINT: Fix linkcheck (mne-tools#12162) ENH: Add multiple label support to source_band_induced_power, source_induced_power (mne-tools#12026) Allow automated metadata generation to be bounded by "row events" instead of explicit time windows (mne-tools#12118) ENH: Collapse only in doc gen (mne-tools#12145) [pre-commit.ci] pre-commit autoupdate (mne-tools#12155) BUG: Fix bug with interior points not showing (mne-tools#12148) ENH: Warn about versions in sys_info (mne-tools#12146) Fix in conftest.py (mne-tools#12150) ENH: set color for bad channel with spatial_colors=True (mne-tools#12142) DOC: Better documentation of realign_raw (mne-tools#12135) Add mne-icalabel wildcard (mne-tools#12143) Remove LGTM.com configuration file (mne-tools#12139) ...

larsoner · 2023-11-04T00:09:20Z

Okay I think this is good enough:

Average loc error: 1.7 mm
Average ori error: 9.6°

Just pushed one tiny commit to separate the fit_dipole output from the print(<dipole results info>) stuff so it's easier to see the average errors in the output.

Ready for review/merge from my end @drammock !

* upstream/main: ENH: Enable sensor-specific OPM coregistration in mne coreg (mne-tools#11405) Tweak README.rst (mne-tools#12166) [pre-commit.ci] pre-commit autoupdate (mne-tools#12177) MAINT: Add branch coverage (mne-tools#12174) OpenSSF (mne-tools#12175) fix docstring in 60_sleep.py (mne-tools#12171) FIX: skip empty lines in read_raw_eyelink (mne-tools#12172)

drammock

looks great! just one nitpick, feel free to roll into a subsequent PR to save CI cycles if you want @larsoner

mne/tests/test_dipole.py

larsoner · 2023-11-14T14:58:57Z

Thanks @JD-Zhu !

Co-authored-by: Eric Larson <larson.eric.d@gmail.com>

JD-Zhu added 2 commits October 12, 2023 18:38

Create __init__.py

1c06f7e

Add KIT phantom data

Create phantom_kit.py

699f659

JD-Zhu added 6 commits October 16, 2023 17:09

Merge branch 'mne-tools:main' into main

79793b2

Update config.py

9f17418

Update utils.py

bc105bb

Update __init__.pyi

c336b47

Update config.py

0498a04

Update config.yml

462a305

JD-Zhu requested a review from larsoner as a code owner October 16, 2023 06:42

Update circleci_download.sh

346d35f

JD-Zhu requested a review from drammock as a code owner October 16, 2023 06:46

Update datasets.rst

62d104d

JD-Zhu requested review from agramfort and dengemann as code owners October 16, 2023 06:48

Update devel.rst

14c03a3

JD-Zhu marked this pull request as draft October 17, 2023 02:13

Update datasets.rst

f135d32

JD-Zhu marked this pull request as ready for review October 17, 2023 02:30

larsoner added 3 commits October 17, 2023 15:19

FIX: Minor fixes

200bfcc

FIX: Correct import check

6b3cdb8

TST: Try reading it

91aa469

larsoner reviewed Oct 17, 2023

View reviewed changes

tutorials/inverse/95_phantom_KIT.py Outdated Show resolved Hide resolved

JD-Zhu and others added 3 commits October 20, 2023 19:03

Update zip file

5482d86

Merge branch 'main' into main

6901598

FIX: Show it

6722c45

JD-Zhu added 3 commits October 21, 2023 13:14

Update docs

81c7974

Merge branch 'main' of https://github.com/JD-Zhu/mne-python into main

fde419e

Update description

9e7bb0b

ENH: Add localization

a5a9bd3

larsoner requested a review from alexrockhill as a code owner October 23, 2023 14:56

larsoner reviewed Oct 23, 2023

View reviewed changes

larsoner added 2 commits October 23, 2023 11:08

FIX: GOF

5be2175

FIX: Link

3090c66

larsoner closed this Oct 27, 2023

larsoner reopened this Oct 27, 2023

larsoner added 5 commits November 3, 2023 19:38

ENH: Add locs and oris

86dffce

FIX: Versionadded

c21cefb

FIX: Compact

aea5d99

FIX: Sections

8210f6f

larsoner added this to the 1.6 milestone Nov 7, 2023

drammock approved these changes Nov 13, 2023

View reviewed changes

mne/tests/test_dipole.py Outdated Show resolved Hide resolved

FIX: Test [ci skip]

88fea0f

larsoner merged commit 16f4411 into mne-tools:main Nov 14, 2023

snwnde pushed a commit to snwnde/mne-python that referenced this pull request Mar 20, 2024

Add KIT phantom dataset (mne-tools#12105)

59baeba

Co-authored-by: Eric Larson <larson.eric.d@gmail.com>

		f"Trigger channel {ch_name} has a non-zero initial value of "
		"{initial_value} (consider using initial_event=True to detect this "

Uh oh!

Add KIT phantom dataset #12105

Add KIT phantom dataset #12105

Uh oh!

Conversation

JD-Zhu commented Oct 12, 2023 • edited by larsoner Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this implement/fix?

Uh oh!

mscheltienne commented Oct 12, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

larsoner commented Oct 12, 2023

Uh oh!

JD-Zhu commented Oct 16, 2023

Uh oh!

larsoner commented Oct 16, 2023

Uh oh!

JD-Zhu commented Oct 17, 2023

Uh oh!

Uh oh!

JD-Zhu commented Oct 21, 2023

Uh oh!

larsoner commented Oct 21, 2023

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

larsoner commented Oct 23, 2023

Uh oh!

JD-Zhu commented Oct 27, 2023

Uh oh!

larsoner commented Oct 27, 2023

Uh oh!

larsoner commented Oct 27, 2023

Uh oh!

larsoner commented Nov 4, 2023

Uh oh!

drammock left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

larsoner commented Nov 14, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

JD-Zhu commented Oct 12, 2023 •

edited by larsoner

Loading

mscheltienne commented Oct 12, 2023 •

edited

Loading