Skip to content

Conversation

@alexrockhill
Copy link
Contributor

@alexrockhill alexrockhill commented Jun 15, 2022

The goal is to abstract the slice browsing for the other GUI that does time-frequency viewing on a volumetric source estimates. Here is a working version of the slice browser itself:

import os.path as op

import numpy as np
import matplotlib.pyplot as plt

import nibabel as nib
import nilearn.plotting
from dipy.align import resample

import mne
from mne.datasets import fetch_fsaverage
from mne.gui._core import SliceBrowser

# paths to mne datasets: sample sEEG and FreeSurfer's fsaverage subject,
# which is in MNI space
misc_path = mne.datasets.misc.data_path()
sample_path = mne.datasets.sample.data_path()
subjects_dir = op.join(sample_path, 'subjects')

# use mne-python's fsaverage data
fetch_fsaverage(subjects_dir=subjects_dir, verbose=True)  # downloads if needed

# GUI requires pyvista backend
mne.viz.set_3d_backend('pyvistaqt')

T1 = nib.load(op.join(misc_path, 'seeg', 'sample_seeg', 'mri', 'T1.mgz'))
CT_orig = nib.load(op.join(misc_path, 'seeg', 'sample_seeg_CT.mgz'))
reg_affine = np.array([
    [0.99270756, -0.03243313, 0.11610254, -133.094156],
    [0.04374389, 0.99439665, -0.09623816, -97.58320673],
    [-0.11233068, 0.10061512, 0.98856381, -84.45551601],
    [0., 0., 0., 1.]])
CT_aligned = mne.transforms.apply_volume_registration(CT_orig, T1, reg_affine)
subj_trans = mne.coreg.estimate_head_mri_t(
    'sample_seeg', op.join(misc_path, 'seeg'))
raw = mne.io.read_raw(op.join(misc_path, 'seeg', 'sample_seeg_ieeg.fif'))

# launch
from qtpy.QtWidgets import QApplication
# get application
app = QApplication.instance()
if app is None:
    app = QApplication(["Slice Browser"])
gui = mne.gui._core.SliceBrowser(CT_aligned, subject='sample_seeg',
                                 subjects_dir=op.join(misc_path, 'seeg'))
gui.show()

Here's what it looks like, which would be inserted into any GUI and then the extra visualization is added on the top/bottom/sides:

Screen Shot 2022-06-15 at 4 43 09 PM

I was thinking for the TFR viewer the spectrograms/time courses could either be to the side split 50/50 with a smaller slice-browser as in here image or between the bottom bar and the slices. Testing changing the window size, keeping proportionality handles much better making the slice browser horizontally narrower so I think option 1 will probably look better. Then buttons to change from power to ITC to time course could be in that panel on the right as well as changing vmin/vmax etc.

I need to:

@alexrockhill alexrockhill changed the title [ENH, WIP] Abstract slice browser [skip ci] [ENH, WIP] Abstract slice browser Jun 15, 2022
@alexrockhill alexrockhill marked this pull request as ready for review June 17, 2022 22:08
@alexrockhill
Copy link
Contributor Author

So I pulled the SliceBrowser out of IntracranialElectrodeLocator in order to use it for another project but in doing so I also renamed things to toolbar and status_bar in preparation to synchronize closer with the abstract Qt backend changes.

I moved over the tests that had to do with navigating slices to test_core.py also.

Here is a minimal script to launch both GUIs:

import os.path as op

import numpy as np
import matplotlib.pyplot as plt

import nibabel as nib
import nilearn.plotting
from dipy.align import resample

import mne
from mne.datasets import fetch_fsaverage
from mne.gui._core import SliceBrowser

# paths to mne datasets: sample sEEG and FreeSurfer's fsaverage subject,
# which is in MNI space
misc_path = mne.datasets.misc.data_path()
sample_path = mne.datasets.sample.data_path()
subjects_dir = op.join(sample_path, 'subjects')

# use mne-python's fsaverage data
fetch_fsaverage(subjects_dir=subjects_dir, verbose=True)  # downloads if needed

# GUI requires pyvista backend
mne.viz.set_3d_backend('pyvistaqt')

T1 = nib.load(op.join(misc_path, 'seeg', 'sample_seeg', 'mri', 'T1.mgz'))
CT_orig = nib.load(op.join(misc_path, 'seeg', 'sample_seeg_CT.mgz'))
reg_affine = np.array([
    [0.99270756, -0.03243313, 0.11610254, -133.094156],
    [0.04374389, 0.99439665, -0.09623816, -97.58320673],
    [-0.11233068, 0.10061512, 0.98856381, -84.45551601],
    [0., 0., 0., 1.]])
CT_aligned = mne.transforms.apply_volume_registration(CT_orig, T1, reg_affine)
subj_trans = mne.coreg.estimate_head_mri_t(
    'sample_seeg', op.join(misc_path, 'seeg'))
raw = mne.io.read_raw(op.join(misc_path, 'seeg', 'sample_seeg_ieeg.fif'))
gui = mne.gui.locate_ieeg(raw.info, subj_trans, CT_aligned,
                          subject='sample_seeg',
                          subjects_dir=op.join(misc_path, 'seeg'))

# launch
from qtpy.QtWidgets import QApplication
# get application
app = QApplication.instance()
if app is None:
    app = QApplication(["Slice Browser"])
gui = mne.gui._core.SliceBrowser(subject='sample_seeg',
                                 subjects_dir=op.join(misc_path, 'seeg'))
gui.show()

@alexrockhill alexrockhill changed the title [ENH, WIP] Abstract slice browser [ENH, MRG] Abstract slice browser Jun 17, 2022
@alexrockhill
Copy link
Contributor Author

@alexrockhill
Copy link
Contributor Author

Failures look unrelated, ready for review when someone has a minute. Then I will work on abstracting the GUI.

@alexrockhill
Copy link
Contributor Author

alexrockhill commented Jun 22, 2022

It looks like there's one issue with pyvistaqt in Python 3.10 still

Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.10.5/x64/lib/python3.10/site-packages/pyvistaqt/rwi.py", line 557, in leaveEvent
    ctrl, shift = self._GetCtrlShift(ev)
  File "/opt/hostedtoolcache/Python/3.10.5/x64/lib/python3.10/site-packages/pyvistaqt/rwi.py", line 515, in _GetCtrlShift
    if ev.modifiers() & KeyboardModifier.ShiftModifier:
  File "/opt/hostedtoolcache/Python/3.10.5/x64/lib/python3.10/enum.py", line 385, in __call__
    return cls.__new__(cls, value)
  File "/opt/hostedtoolcache/Python/3.10.5/x64/lib/python3.10/enum.py", line 718, in __new__
    raise exc
  File "/opt/hostedtoolcache/Python/3.10.5/x64/lib/python3.10/enum.py", line 700, in __new__
    result = cls._missing_(value)
  File "/opt/hostedtoolcache/Python/3.10.5/x64/lib/python3.10/enum.py", line 846, in _missing_
    possible_member = cls._create_pseudo_member_(value)
  File "/opt/hostedtoolcache/Python/3.10.5/x64/lib/python3.10/enum.py", line 861, in _create_pseudo_member_
    raise ValueError("%r is not a valid %s" % (value, cls.__qualname__))
ValueError: 1662316080 is not a valid Qt.KeyboardModifier
Fatal Python error: Aborted

I think it's only running on this PR because things were changed? Not sure though, because in other CIs on other PRs it's just skipped. @larsoner, would you happen to know anything about this?

See for instance:

ne/gui/tests/test_coreg_gui.py::test_coreg_gui_notebook[notebook] SKIPPED (Skipping Notebook test: No module named 'ipywidgets') [ 15%]

from here https://github.com/mne-tools/mne-python/runs/6993388950?check_suite_focus=true#step:13:688

@alexrockhill
Copy link
Contributor Author

Hey, finally looks like it'll pass all the tests! Ready to merge I think unless anyone wants changes

@alexrockhill
Copy link
Contributor Author

If anyone has a minute to review, it would be nice to continue on the core deliverables of my GSoC project which depends on this PR so it's a bit of a bottleneck at the moment @agramfort @britta-wstnr @larsoner

@larsoner
Copy link
Member

@alexrockhill can you rebase or merge with main? Looks like there is a conflict now

We can ignore macOS as that's #10805 / #10820

Copy link
Member

@larsoner larsoner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other than the conflict, one high-level idea (probably for later), and one question, LGTM!

return canvas, fig


class SliceBrowser(QMainWindow):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

More future compatible perhaps to make this a QWidget? Then it can be embedded in any main window, popup, etc. We could do this in a separate PR, though, if that makes more sense / keeps the diff smaller (which it might, since the iEEG GUI code now just mostly wraps to this)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm this is a good point but then it won't pop up in a main window for its own tests. We could easily wrap it though. Conceptually, it does make sense as a widget. Do you mind if I push this to the next PR when I put it in the TFR source space browser? I think also, I've been working on the abstraction code and that might effect it so it might be better to address in parts.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah a follow-up PR is fine

but then it won't pop up in a main window for its own tests.

Have you tried? IIRC Qt might instantiate a window for you if you do this, and even if it doesn't actually pop up, I think that's okay for tests anyway. We just need to make sure the interactions / signals etc. actually work, which we do non-visually anyway

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

... okay at least creating some simple widgets it did not show. But that might end up being okay in testing. If it's not, I agree some simple wrapper (probably a pytest fixture that returns a window -- pytest-qt might even have this already?) would make sense.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm interested to try, also ideally would be a notebook widget too, hopefully soon

@@ -0,0 +1,483 @@
# -*- coding: utf-8 -*-
"""Shared GUI classes and functions."""
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not look at this file much at all because I assume it is 99% copy-paste from the GUI code in main. Is that correct? If not, can you point out which bits of the code are new in this file that I should focus on?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes that is correct, I just tried to copy-paste and move all the code that is specific to slice browsing to the core.py file with a few small changes to make it work.

fix mr vs ct handling [skip ci]

wip

revert qt abstraction changes, finished with slice browser abstraction

integrate with ieeg_locate, add tests

fix flake, fix tests?

fix codespell

test fix again

fix style

fix test

one more try

fix skip tests

another try for skiptest

try again with pyvistaqt in skip

oh, I think I fixed it, just missing a conftest arg

use fixture, hopefully fixes issues
@alexrockhill
Copy link
Contributor Author

Sorry for spamming emails with futile commits. When I try to build the docs locally, I get the following over and over presumably because the process failed and is a zombie, does anyone know how to fix this?

BUILD_DEV_HTML=1 sphinx-build -D sphinx_gallery_conf.filename_pattern= -D sphinx_gallery_conf.run_stale_examples=True -b html -d _build/doctrees  -nWT --keep-going . _build/html
Running Sphinx v4.0.2
Using pyvistaqt 3d backend.

Using qt as 2D backend.
Building documentation for MNE 1.1.dev0 (/Users/alexrockhill/projects/mne-python/mne/__init__.py)
Building with scrapers=('matplotlib', <GUIScraper>, <BrainScraper>, 'pyvista', <ReportScraper>, <MNEQtBrowserScraper>)
checking for /Users/alexrockhill/projects/mne-python/doc/references.bib in bibtex cache... not found
parsing bibtex file /Users/alexrockhill/projects/mne-python/doc/references.bib... parsed 204 entries
[autosummary] generating autosummary for: auto_tutorials/intro/10_overview.rst, bibliography.rst, cited.rst, connectivity.rst, covariance.rst, creating_from_arrays.rst, datasets.rst, decoding.rst, events.rst, export.rst, ..., reading_raw_data.rst, realtime.rst, report.rst, sensor_space.rst, simulation.rst, source_space.rst, statistics.rst, time_frequency.rst, visualization.rst, whats_new.rst
loading intersphinx inventory from https://docs.python.org/3/objects.inv...
loading intersphinx inventory from https://numpy.org/devdocs/objects.inv...
loading intersphinx inventory from https://scipy.github.io/devdocs/objects.inv...
loading intersphinx inventory from https://matplotlib.org/stable/objects.inv...
loading intersphinx inventory from https://scikit-learn.org/stable/objects.inv...
loading intersphinx inventory from https://numba.pydata.org/numba-doc/latest/objects.inv...
loading intersphinx inventory from https://joblib.readthedocs.io/en/latest/objects.inv...
loading intersphinx inventory from https://nipy.org/nibabel/objects.inv...
loading intersphinx inventory from http://nilearn.github.io/objects.inv...
loading intersphinx inventory from https://nipy.org/nitime/objects.inv...
WARNING: failed to reach any of the inventories with the following issues:
intersphinx inventory 'http://nilearn.github.io/objects.inv' not fetchable due to <class 'requests.exceptions.HTTPError'>: 404 Client Error: Not Found for url: http://nilearn.github.io/objects.inv
loading intersphinx inventory from https://pysurfer.github.io/objects.inv...
loading intersphinx inventory from https://mne.tools/mne-bids/stable/objects.inv...
loading intersphinx inventory from https://mne.tools/mne-connectivity/stable/objects.inv...
loading intersphinx inventory from https://pandas.pydata.org/pandas-docs/stable/objects.inv...
loading intersphinx inventory from https://seaborn.pydata.org/objects.inv...
loading intersphinx inventory from https://www.statsmodels.org/dev/objects.inv...
loading intersphinx inventory from https://patsy.readthedocs.io/en/latest/objects.inv...
loading intersphinx inventory from https://docs.pyvista.org/objects.inv...
loading intersphinx inventory from https://imageio.readthedocs.io/en/latest/objects.inv...
loading intersphinx inventory from https://mne.tools/mne-realtime/objects.inv...
loading intersphinx inventory from https://pierreablin.github.io/picard/objects.inv...
loading intersphinx inventory from https://qdarkstylesheet.readthedocs.io/en/latest/objects.inv...
loading intersphinx inventory from https://eeglabio.readthedocs.io/en/latest/objects.inv...
loading intersphinx inventory from https://dipy.org/documentation/latest/objects.inv/...
loading intersphinx inventory from https://www.fatiando.org/pooch/latest/objects.inv...
loading intersphinx inventory from https://pybv.readthedocs.io/en/latest/objects.inv...
intersphinx inventory has moved: https://dipy.org/documentation/latest/objects.inv/ -> https://dipy.org/documentation/1.5.0/objects.inv/
generating gallery...

Traceback (most recent call last):
  File "/Users/alexrockhill/software/anaconda3/envs/mne/lib/python3.9/site-packages/psutil/_common.py", line 441, in wrapper
    ret = self._cache[fun]
AttributeError: 'Process' object has no attribute '_cache'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/alexrockhill/software/anaconda3/envs/mne/lib/python3.9/site-packages/psutil/_common.py", line 441, in wrapper
    ret = self._cache[fun]
AttributeError: _cache

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/alexrockhill/software/anaconda3/envs/mne/lib/python3.9/site-packages/psutil/_psosx.py", line 343, in wrapper
    return fun(self, *args, **kwargs)
  File "/Users/alexrockhill/software/anaconda3/envs/mne/lib/python3.9/site-packages/psutil/_common.py", line 444, in wrapper
    return fun(self)
  File "/Users/alexrockhill/software/anaconda3/envs/mne/lib/python3.9/site-packages/psutil/_psosx.py", line 378, in _get_pidtaskinfo
    ret = cext.proc_pidtaskinfo_oneshot(self.pid)
ProcessLookupError: [Errno 3] No such process (originated from proc_pidinfo())

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/alexrockhill/.local/lib/python3.9/site-packages/sphinx/events.py", line 101, in emit
    results.append(listener.handler(self.app, *args))
  File "/Users/alexrockhill/software/anaconda3/envs/mne/lib/python3.9/site-packages/sphinx_gallery/gen_gallery.py", line 426, in generate_gallery_rst
    gallery_conf = parse_config(app)
  File "/Users/alexrockhill/software/anaconda3/envs/mne/lib/python3.9/site-packages/sphinx_gallery/gen_gallery.py", line 121, in parse_config
    gallery_conf = _complete_gallery_conf(
  File "/Users/alexrockhill/software/anaconda3/envs/mne/lib/python3.9/site-packages/sphinx_gallery/gen_gallery.py", line 202, in _complete_gallery_conf
    gallery_conf['memory_base'] = _get_memory_base(gallery_conf)
  File "/Users/alexrockhill/software/anaconda3/envs/mne/lib/python3.9/site-packages/sphinx_gallery/gen_rst.py", line 553, in _get_memory_base
    memories = memory_usage(proc, interval=1e-3, timeout=timeout)
  File "/Users/alexrockhill/.local/lib/python3.9/site-packages/memory_profiler.py", line 362, in memory_usage
    mem_usage = _get_memory(
  File "/Users/alexrockhill/.local/lib/python3.9/site-packages/memory_profiler.py", line 184, in _get_memory
    return tools[backend]()
  File "/Users/alexrockhill/.local/lib/python3.9/site-packages/memory_profiler.py", line 134, in ps_util_tool
    mem = getattr(process, meminfo_attr)()[0] / _TWO_20
  File "/Users/alexrockhill/software/anaconda3/envs/mne/lib/python3.9/site-packages/psutil/_common.py", line 444, in wrapper
    return fun(self)
  File "/Users/alexrockhill/software/anaconda3/envs/mne/lib/python3.9/site-packages/psutil/__init__.py", line 1061, in memory_info
    return self._proc.memory_info()
  File "/Users/alexrockhill/software/anaconda3/envs/mne/lib/python3.9/site-packages/psutil/_psosx.py", line 343, in wrapper
    return fun(self, *args, **kwargs)
  File "/Users/alexrockhill/software/anaconda3/envs/mne/lib/python3.9/site-packages/psutil/_psosx.py", line 443, in memory_info
    rawtuple = self._get_pidtaskinfo()
  File "/Users/alexrockhill/software/anaconda3/envs/mne/lib/python3.9/site-packages/psutil/_psosx.py", line 348, in wrapper
    raise NoSuchProcess(self.pid, self._name)
psutil.NoSuchProcess: process no longer exists (pid=80081)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/alexrockhill/.local/lib/python3.9/site-packages/sphinx/cmd/build.py", line 276, in build_main
    app = Sphinx(args.sourcedir, args.confdir, args.outputdir,
  File "/Users/alexrockhill/.local/lib/python3.9/site-packages/sphinx/application.py", line 276, in __init__
    self._init_builder()
  File "/Users/alexrockhill/.local/lib/python3.9/site-packages/sphinx/application.py", line 335, in _init_builder
    self.events.emit('builder-inited')
  File "/Users/alexrockhill/.local/lib/python3.9/site-packages/sphinx/events.py", line 109, in emit
    raise ExtensionError(__("Handler %r for event %r threw an exception") %
sphinx.errors.ExtensionError: Handler <function generate_gallery_rst at 0x13ba3c4c0> for event 'builder-inited' threw an exception (exception: process no longer exists (pid=80081))

Extension error (sphinx_gallery.gen_gallery):
Handler <function generate_gallery_rst at 0x13ba3c4c0> for event 'builder-inited' threw an exception (exception: process no longer exists (pid=80081))
Exception ignored in: <function Popen.__del__ at 0x103deeaf0>
Traceback (most recent call last):
  File "/Users/alexrockhill/software/anaconda3/envs/mne/lib/python3.9/subprocess.py", line 1052, in __del__
    _warn("subprocess %s is still running" % self.pid,
ResourceWarning: subprocess 80081 is still running
make: *** [html_dev-pattern] Error 2

@larsoner
Copy link
Member

On macOS psutil/memory_profiler are a bit unreliable :( I'm not sure of a good fix or workaround

@larsoner
Copy link
Member

... actually one workaround would be to set locally profile_memory=False in doc/conf.py. And if you really want to know the memory usage, locally do:

memprof run tutorials/clinical/whatever.py && mprof plot

it should pop up a plot of memory usage

@larsoner
Copy link
Member

And thinking about it more +1 for changing 'show_memory': not sys.platform.startswith('win') to 'show_memory': not sys.platform.startswith(('win', 'darwin')) in conf.py

@alexrockhill
Copy link
Contributor Author

alexrockhill commented Jun 24, 2022

And thinking about it more +1 for changing 'show_memory': not sys.platform.startswith('win') to 'show_memory': not sys.platform.startswith(('win', 'darwin')) in conf.py

Ok, I can do a PR. Thanks!

@alexrockhill
Copy link
Contributor Author

So I guess the warning filters in conftest.py do not apply to Circle... do you know how to fix the PendingDepreciationWarning?

@alexrockhill
Copy link
Contributor Author

Okay to merge? Failures are unrelated

@larsoner
Copy link
Member

Let's get a full CI build now that they should succeed

@larsoner
Copy link
Member

I restarted circle, let's merge if that comes back green

@larsoner
Copy link
Member

Thanks @alexrockhill

@larsoner larsoner merged commit da02a25 into mne-tools:main Jun 24, 2022
@alexrockhill alexrockhill deleted the slice branch June 24, 2022 22:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants