New epochs class and marker mapping

[qian-chu]

Epochs

Developers and users have found the old Epochs class quite unintuitive and operationally hard to use (e.g., nested DataFrame). This PR aims to reorganize the class.

Current features:

• Slight change of Events class, where now event IDs (e.g., "fixation id") are now used as index. For messages and custom events, the index is name "event id"
• event_times is renamed to epochs_info for easier comprehension
• Creating an Epochs instance will not compute the epochs or annotated data automatically.
• epochs is now a cached property in the form of a dictionary. Keys are indices of epochs_info and values are Stream/Events instances.
• Annotated data now available through annotate_epochs.
• (in progress) added a pupillary light reflex (PLR) dataset (2 participants, data in cloud and native formats). Currently exploring options to host data on FigShare vs OSF

Marker mapping

Per suggestions, this PR also aims to expand the marker mapping from AprilTags to also ArUco markers (using OpenCV's aruco module). This direct interface also allows us to get rid of the optional dependency of pupil-apriltags. API and documentations are improved along the way.

[qian-chu]

@JGHartel Currently the baseline correction method is not updated yet. What do you think would be the best API/code for it? For operability with other methods such as plot(), I think it makes the most sense to modify the data in epochs property

[JGHartel]

Overall, I welcome the suggested changes. In particular, the rename towards epoch_info and the unified access to epochs data as epoch.epochs, rather than the previous mixed access, is appreciated. Still, I would like to invite some discussions on the assumed datatypes, e.g. epochs.epochs and epochs.annotate being a dictionary. This would especially futureproof the development, if dataset level operations should be introduced

[JGHartel]

flagging this in case we ever want to support multi-recording functionality. It would be good to have a Unique ID for each event, so that one can easily concat these dicts between recordings. Alternatively, we could then construct an implicit multiindex (recording, epoch_number), which would require a custom concat function.

[qian-chu]

This seems a bit of an advanced user case and also in principle should apply to other classes (e.g. streams and events). My evaluation would be it would require another PR if we see value in supporting multi-recording concatenation

[Copilot]

Pull request overview

This PR introduces a major refactoring of the Epochs class to make it more intuitive and easier to use. The key changes include moving away from nested DataFrames to a dictionary-based structure, renaming methods for clarity, and introducing lazy computation of epochs using cached properties.

Changes:

• Refactored Epochs class to use epochs_dict (cached property returning a dictionary) instead of nested DataFrames
• Renamed event_times to epochs_info and events_to_times_df to events_to_epochs_info for better comprehension
• Changed Events class to use event IDs as DataFrame index instead of separate columns
• Added new filter_by_name method to Events class and updated filter_by_duration behavior
• Added PLR sample dataset support with Figshare hosting
• Consolidated CI workflows (removed separate tests.yml, integrated into main.yml)

Reviewed changes

Copilot reviewed 19 out of 22 changed files in this pull request and generated 17 comments.

Show a summary per file

File	Description
pyneon/epochs.py	Major refactoring of Epochs class with new dictionary-based structure, cached properties, and helper functions renamed
pyneon/events.py	Modified to use event IDs as index, added filter_by_name method, updated filter_by_duration
pyneon/stream.py	Added stub for detect_events_from_derivative method, updated docstrings with reusable doc snippets
pyneon/vis/vis.py	Updated plot_epochs to work with new epochs_dict structure
pyneon/dataset.py	Made sections.csv optional, improved error handling for missing recordings
pyneon/utils/doc_decorators.py	Added reusable documentation snippets for common return types and parameters
pyneon/utils/sample_data.py	Added PLR dataset URL and test
tests/conftest.py	Added simple_events fixture for testing Events functionality
tests/test_events.py	Added (empty) test stub for crop functionality
tests/test_streams.py	Added tests for interpolation and concatenation methods
.github/workflows/main.yml	Consolidated CI workflows, added test job dependencies
source/tutorials/*.ipynb	Updated tutorial notebooks to use new API names
README.md	Added information about sample datasets on Figshare

Comments suppressed due to low confidence (1)

pyneon/epochs.py:223

• The properties columns and dtypes (lines 216-223) reference self.data which no longer exists in the new Epochs class structure. These properties need to be updated or removed.

    @property
    def columns(self) -> pd.Index:
        return self.data.columns[:-3]

    @property
    def dtypes(self) -> pd.Series:
        """The data types of the epoched data."""
        return self.data.dtypes[:-3]

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.