Issue with cca_crossvalidate

[nbara]

AA, RR, SD = cca.cca_crossvalidate(raw_fin, env_fin, shifts=None, sfreq=1, surrogate=False, plot=False) outputs an empty AA matrix.

[cpelofi]

Hi there,
quick question about the thresh value in whiten_nt that is set to thresh=1e-12.
What does it corresponds to? it seems that during the whitening step, line 386:
dd[d > thresh] = (1. / d[d > thresh])

none of my values meet the threshold and I end up with an empty array.
Thanks in advance for your help!

[nbara]

Hey, this is used to discard weak principal components in the PCA.

Is your data in T by any chance? Have you tried inputting your data in fT instead?

If that doesn't work can you send over a small file that I could use for testing ?

[cpelofi]

I am trying to upload the .npz file but it does not seem to be an accepted format

[nbara]

Just change the extension to .zip, or zip the file. 

…

--Nicolas Sent from my phone 

[cpelofi]

data_2trials.npz.zip

[cpelofi]

Hi there,

Thanks for your help, here the data as it is input into the cca.cca_crossvalidate function, only with 2 trials instead of 10.

[nbara]

Hi Claire,

As I suspected this issue was occurring because your data is very small (10 ^ -14, presumably in Tesla).

The code below produced seemingly reasonable matrices when I scaled the data back to fT (raw * 1e15):

data = np.load('./tests/data/ccadata_meg_2trials.npz')
raw = data['arr_0']
env = data['arr_1']

# Test with scaling (unit: fT)
A0, B0, R0 = nt_cca(raw * 1e15, env)  # provides reasonable matrices

# Test without scaling (unit: T)
A1, B1, R1 = nt_cca(raw, env)  # outputs all-zero matrices

np.testing.assert_almost_equal(R0, R1)  # does not work, so the scaling has an effect

There was an issue with how the whitening was done though (for very low amplitude data like this), and I pushed a fix for this here. You can try that code to see if it solves your problem.

I would however advise you to work with fT whenever possible, not just in this case but in general in order to avoid numerical instabilities in your processing pipeline.

[cpelofi]

Got it! I have come across some scaling problems with MEG data so this makes sense.
thanks for your help!!

[nbara]

No problem!