BUG: Fix bug related to how Neuroscan .cnt n_samples is read

[withmywoessner]

Reference issue

#6537
#11802

What does this implement/fix?

Change how number of samples is read and verifies that they are correct.

Additional information

I need to upload new testing data that demonstrates this bug.

[withmywoessner]

@larsoner Can I update the testing data to include a file that was not read properly before this change or even replace the one I already added.

The documentation said that the header I am using was not reliable, but I think that may be because they read the n_samples as normal int when it should be an unsigned int. The eeglab load_cnt function reads the n_samples properly and they also read it as an unsigned int.

I am not sure why this broke a bunch of tests. The cnt.py tests ran fine and I plotted all the testing data and it all looks fine.

[larsoner]

Can I update the testing data to include a file that was not read properly before this change or even replace the one I already added.

Yes feel free! The file is small, right? The smaller the testing dataset the better, so might as well replace the old one assuming we can still test all the same stuff.

[withmywoessner]

@larsoner I added a check to see if there were any annotations outside the actual amount of data. If there are, I switch the data_format. I could not figure out a way to detect the correct number of samples with certainty. It only triggers when using the 'auto' data_format which already doesn't work with the current testing data I uploaded. Is this fine with you?

[withmywoessner]

For some reason I was unable to run my test locally using pytest

[drammock]

Test function names must start with "test_" yours starts with "verify_"

…

-------- Original Message --------

On 1/29/24 16:09, Jacob Woessner wrote: @withmywoessner commented on this pull request. --------------------------------------------------------------- In [mne/io/cnt/tests/test_cnt.py](#12393 (comment)): > ***@***.***_testing_data +def verify_reading_bytes(): + raw_16 = read_raw_cnt(fname, preload=True) + raw_32 = read_raw_cnt(fname_bad_spans, preload=True) + + # Verify that the number of bytes read is correct + assert len(raw_16) == 3070 + assert len(raw_32) == 90000 + + For some reason I was unable to run my test locally using pytest — Reply to this email directly, [view it on GitHub](#12393 (review)), or [unsubscribe](https://github.com/notifications/unsubscribe-auth/AAN2AU44A5WG67QGJHTECQ3YRAMXZAVCNFSM6AAAAABCOVJKUCVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMYTQNBZHA3DOOBZGM). You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[larsoner]

And don't forget to change the hash and version number of the testing dataset in mne/datasets/config.py

[withmywoessner]

Done! @larsoner New hash and version is in the config file:

Everything should be good to go! Thanks!

[larsoner]

CIs are unhappy, can you look?

[withmywoessner]

CIs are unhappy, can you look?

I forgot a docustring. It should hopefully be good @larsoner. I think this is causing it to fail now:

mne-python/mne/io/cnt/tests/test_cnt.py

Line 23 in 1ffe426

_no_parse = pytest.warns(RuntimeWarning, match="Could not parse")

[larsoner]

Looking at the logs the error is:

mne/io/cnt/tests/test_cnt.py:57: in test_auto_data
    with pytest.warns(RuntimeWarning, match="Omitted 6 annot"):
E   RuntimeWarning: Could not define the number of bytes automatically. Defaulting to 2.

So I think you need to change the test now to catch a different warning string (or maybe multiple warning strings with multiple pytest.warns)

[withmywoessner]

Looking at the logs the error is:

mne/io/cnt/tests/test_cnt.py:57: in test_auto_data
    with pytest.warns(RuntimeWarning, match="Omitted 6 annot"):
E   RuntimeWarning: Could not define the number of bytes automatically. Defaulting to 2.

So I think you need to change the test now to catch a different warning string (or maybe multiple warning strings with multiple pytest.warns)

Oh, I thought it was the _no_parse test causing it to fail. I saw this when looking at the log:

I removed it and they pass now. Maybe I missed something? @larsoner

[larsoner]

Are you sure tests are passing and not being skipped at your end? In your mne-testing-data PR you had an outdated version in your version.txt locally, maybe that's causing a problem. Or maybe you're not on pytest>=8 where they changed the warning catching behavior. In any case, I could replicate the error locally and pushed a commit to fix it.

[larsoner]

@withmywoessner ready to go from your end? I clicked the "ready for review" button in case but let me know either way (and no need to put [wip] in your title, it's clear from draft status!)

[withmywoessner]

Or maybe you're not on pytest>=8 where they changed the warning catching behavior. In any case, I could replicate the error locally and pushed a commit to fix it.

This fixed it. Thank you!

[withmywoessner]

@withmywoessner ready to go from your end? I clicked the "ready for review" button in case but let me know either way (and no need to put [wip] in your title, it's clear from draft status!)

I am @larsoner. Thanks!

[larsoner]

Just one minor tweak that I'll commit, thanks @withmywoessner !