[MRG] Add events/annotations tutorial #5608

massich · 2018-10-15T13:47:40Z

From the whish list in #5201 it mainly addresses point 4.

codecov · 2018-10-17T09:39:03Z

Codecov Report

Merging #5608 into master will increase coverage by 0.01%.
The diff coverage is 89.47%.

@@            Coverage Diff             @@
##           master    #5608      +/-   ##
==========================================
+ Coverage   88.55%   88.57%   +0.01%     
==========================================
  Files         369      369              
  Lines       69182    69194      +12     
  Branches    11653    11654       +1     
==========================================
+ Hits        61266    61286      +20     
+ Misses       5060     5056       -4     
+ Partials     2856     2852       -4

larsoner · 2018-11-05T20:20:44Z

@massich do we need this for 0.17? If so it's likely to be one of the last blockers for release

massich · 2018-11-06T13:22:23Z

~~yes, we need this one.~~ And goes after #5695

It is nice to have. you moved #5201 to 0.18 already. These 2 go together.

massich · 2018-11-09T17:58:16Z

From the whish list in #5201 it mainly addresses point 4 which is the urgent part to get in 0.17. (So I'll edit the PR description)

agramfort · 2018-11-09T18:05:21Z

rendered https://10162-1301584-gh.circle-artifacts.com/0/html/auto_tutorials/plot_events_and_annotations.html

massich · 2018-11-09T18:12:30Z

There's still one broken link.

massich · 2018-11-09T18:14:35Z

I don't understand why this link does not work and it does not appear in the build as broken

ref:`sphx_glr_auto_tutorials_plot_artifacts_correction_rejection.py`

massich · 2018-11-09T18:18:43Z

its the same as this is linking to. But here renders properly. I don't get it.

larsoner · 2018-11-09T19:53:53Z

tutorials/plot_events_and_annotations.py

@@ -0,0 +1,134 @@
+"""
+.. _tut_events_and_annotation_objects:


remove this label, we do not need it (it's in other examples for historical reasons and those should be removed, too)

we can use the auto-generated sphx_glr_... labels instead

larsoner · 2018-11-09T20:05:47Z

tutorials/plot_events_and_annotations.py

+Events and annotations are quite similar. This tutorial highlights their
+differences and similitudes and tries to shade some light to which one is
+preferred to use in different situations when using MNE.
+Here follows both terms definition from the glossary.


Would be good to add a RST link to the glossary

yes but I did not find how. I don't think is referenceable yet.

jasmainak · 2018-11-10T13:44:05Z

tutorials/plot_events_and_annotations.py

+    events
+        Events correspond to specific time points in raw data; e.g., triggers,
+        experimental condition events, etc. MNE represents events with integers
+        that are stored in numpy arrays of shape (n_events, 3). Such arrays are


I am always forget what the second column of the events array is. Maybe it's worth repeating here what the three columns represent?

For this you need an example where the second column is more than array of 0s. And is not yet covered in this tutorial.

jasmainak · 2018-11-10T13:44:41Z

tutorials/plot_events_and_annotations.py

+        Events correspond to specific time points in raw data; e.g., triggers,
+        experimental condition events, etc. MNE represents events with integers
+        that are stored in numpy arrays of shape (n_events, 3). Such arrays are
+        classically obtained from a trigger channel, also referred to as stim


stim is really an abbreviation. Since it's a tutorial, I'd suggest expanding to the full name

My only goal here was just to replicate whatever is in glossary. Once glossary is linkable I would remove it.

jasmainak · 2018-11-10T13:45:09Z

tutorials/plot_events_and_annotations.py

+        classically obtained from a trigger channel, also referred to as stim
+        channel.
+
+    annotations


Link to documentation of annotations class?

It is linked everywhere, but here 'cos this is a citation of the glossary:

massich · 2018-11-12T16:41:56Z

I think I've addressed everybody's concerns. What I did not manage is to link to the annotations attributes. (ie: :attr:orig_time <mne.Annotations.orig_time>) To do so, the Annotations parameters should be replicated under the title Attributres ('cos we need them as Parameters and Attributes) but even then the corssreferencing was not working for me. So I gave up. If someone is more skilled than me please push in this branch or make a PR in my fork.

@larsoner yes, they are inconsistent. I did not check if the inconsistency was released already. But I was certainly planning to correct it after the release due to #5699. And maybe find a better name for _handle_meas_date and expose it.

Please, give a read to the tutorial. Thx a lot.

larsoner

Otherwise LGTM

larsoner · 2018-11-12T17:12:47Z

tutorials/plot_object_annotations.py

+
+Both events and :class:`Annotations <mne.Annotations>` be seen as triplets
+where the first element answers to **when** something happens and the last
+element refers to **what** is it.


larsoner · 2018-11-12T17:13:55Z

tutorials/plot_object_annotations.py

+the description is an integer value.
+In contrast, :class:`Annotations <mne.Annotations>` represents the
+``onset`` in seconds (relative to the reference ``orig_time``),
+ and the ``description`` is an arbitrary string.


Remove leading space

jasmainak · 2018-11-12T18:40:28Z

tutorials/plot_object_annotations.py

+# Create an annotation object with orig_time
+orig_time = '2002-12-03 19:01:31.676071'
+my_datetime = datetime.strptime(orig_time, '%Y-%m-%d %H:%M:%S.%f')
+posix_timestamp = (my_datetime - datetime(1970, 1, 1)).total_seconds()


this is not used in the annotations object

No I used only for printing. There's a subtiliy. the orig_time has to be .6760709 otherwise they wont match.

well okay but you don't use this orig_time either ...

print(posix_timestamp - annot_orig.orig_time)

1.1920928955078125e-07

I am using the same number. (except the rounding error)

1038942091.676071 is the POSIX timestamp of 2002-12-03 19:01:31.676071

it only differs in the .6760709

okay maybe then it's a matter of renaming your variables. Because I see this:

orig_time = '2002-12-03 19:01:31.676071'

and I'm confused why in the class instantiation you do: orig_time=1038942091.6760709.

also how much does the 1e-7 matter? couldn't we live with a slightly offset annotation with 1 sample?

@jasmainak I agree it's probably too much to ask to our user to understand this gymnastic of dates.
I am going to push now a way to pass directly '2002-12-03 19:01:31.676071' to the Annotations constructor.

actually I need #5699 first to do this... let me switch to #5699 then.

jasmainak · 2018-11-12T18:41:25Z

tutorials/plot_object_annotations.py

+
+annot = mne.Annotations(onset=[10], duration=[0.5],
+                        description=['foobar'],
+                        orig_time=1038942091.6760709)


still to be fixed ...

I don't see this timestamp as a problem. At least now it is stated that it has been 'chosen' and that is not completely arbitrary. But shows that orig_time is under the hood this kind of ugly floats.

Sorry to insist here.

But if a user wanted to use the Annotations class, how do you propose that they would set orig_time? I don't think anyone could dream up this number. If the idea to expose this parameter is that allows people to adjust the orig_time if it's not aligned. You could perhaps make it annot.orig_time - 0.2 to shift orig time ...

jasmainak

Except those last two comments LGTM. Thanks @massich !

massich · 2018-11-12T21:39:21Z

Onsets no longer match:

[42.95597088 44.95597088 51.95597088]
[42.955971 44.955971 51.955971]

We need to be able to parse orig_time until there's no difference in the delta time.
We should be able to do orig_time='2002-12-03 19:01:31.6760709' and it should take it. There should be a manner to tweak the strptime regex '%Y-%m-%d %H:%M:%S.%f' to take it. I did try when writting a9c0e66 I guess that I've to try harder. We'll find it.

agramfort · 2018-11-12T23:05:59Z

we should round at the microsecond

larsoner · 2018-11-13T00:55:24Z

I pushed a commit that at least makes things pass on both systems. Had to use assert_allclose though. Maybe rounding is better

larsoner · 2018-11-13T00:56:31Z

(I'm talking about pushing ee812c8, so over on #5699)

larsoner · 2018-11-13T02:23:19Z

Pushed a better version that did not necessitate using assert_allclose (760640e)

Done via ISO8601

massich · 2018-11-13T13:02:11Z

It should be able to go in once green.

larsoner · 2018-11-13T15:09:37Z

@jasmainak feel free to merge when you are happy

jasmainak · 2018-11-13T15:29:51Z

Looks really nice. Thanks @massich and @agramfort for the teamwork !

agramfort · 2018-11-13T16:06:45Z

great !

mmagnuski · 2018-11-13T17:21:15Z

tutorials/plot_object_annotations.py

+annotated_blink_raw = raw.copy()
+eog_events = mne.preprocessing.find_eog_events(raw)
+n_blinks = len(eog_events)
+# Center to cover the whole blink with full duration of 0.5s:


this may be a bit misleading - the events are centered and we are actually de-cenetering them by pushing them back in time to mark onset, not event center.

agramfort · 2018-11-13T20:17:41Z

please send a quick PR.

mmagnuski · 2018-11-13T20:22:30Z

@agramfort sure thing, I have some other small suggestions. :)

agramfort · 2018-11-13T20:35:29Z

go for it !

massich changed the title ~~Add tutorial~~ [WIP] Add events/annotations tutorial Oct 15, 2018

massich force-pushed the annotation_tutorial branch from b2dc85f to 180620d Compare October 17, 2018 09:38

larsoner added this to the 0.17 milestone Nov 5, 2018

massich modified the milestones: 0.17, 0.18 Nov 6, 2018

Add tutorial

80c6966

massich force-pushed the annotation_tutorial branch from 48a7308 to 80c6966 Compare November 9, 2018 13:02

Joan Massich added 6 commits November 9, 2018 16:19

illustrate orig_time

ec6d9ad

Add orig_time in annotations __repr__

c63b559

clean up

5e83507

clean up

c82b367

more clean up

ac146d5

Add whatsnew

06c715e

massich changed the title ~~[WIP] Add events/annotations tutorial~~ [MRG] Add events/annotations tutorial Nov 9, 2018

fix broken link

70e8a80

larsoner reviewed Nov 9, 2018

View reviewed changes

Joan Massich added 2 commits November 10, 2018 13:01

remove old link reference

0fa243d

pep8

cdb1183

jasmainak reviewed Nov 10, 2018

View reviewed changes

massich force-pushed the annotation_tutorial branch from b18b962 to a9c0e66 Compare November 12, 2018 16:33

larsoner reviewed Nov 12, 2018

View reviewed changes

FIX: Minor fixes

7622844

larsoner approved these changes Nov 12, 2018

View reviewed changes

jasmainak reviewed Nov 12, 2018

View reviewed changes

jasmainak previously requested changes Nov 12, 2018

View reviewed changes

agramfort and others added 4 commits November 12, 2018 21:57

pass on text

f291830

Add str support to _handle_meas_date

cce8cb7

allow passing orig_time in ISO8601 format

3901065

Migrate whatever was missing in _handle_meas_date form mne-tools#5699

174d94a

print 6 decimals

60912ed

larsoner approved these changes Nov 13, 2018

View reviewed changes

jasmainak merged commit df832f4 into mne-tools:master Nov 13, 2018

massich deleted the annotation_tutorial branch November 13, 2018 15:52

mmagnuski reviewed Nov 13, 2018

View reviewed changes

Uh oh!

[MRG] Add events/annotations tutorial #5608

[MRG] Add events/annotations tutorial #5608

Uh oh!

Conversation

massich commented Oct 15, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented Oct 17, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

larsoner commented Nov 5, 2018

Uh oh!

massich commented Nov 6, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

massich commented Nov 9, 2018

Uh oh!

agramfort commented Nov 9, 2018

Uh oh!

massich commented Nov 9, 2018

Uh oh!

massich commented Nov 9, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

massich commented Nov 9, 2018

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jasmainak Nov 10, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

massich commented Nov 12, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

larsoner left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jasmainak Nov 12, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

massich commented Oct 15, 2018 •

edited

Loading

codecov bot commented Oct 17, 2018 •

edited

Loading

massich commented Nov 6, 2018 •

edited

Loading

massich commented Nov 9, 2018 •

edited

Loading

jasmainak Nov 10, 2018 •

edited

Loading

massich commented Nov 12, 2018 •

edited

Loading

jasmainak Nov 12, 2018 •

edited

Loading

massich commented Nov 12, 2018 •

edited

Loading

larsoner commented Nov 13, 2018 •

edited

Loading