Applying ZapLine to continuous data

[marcpabst]

Hi, thanks for your package.

I was wondering if you suggest segmenting continuous data before applying the ZapLine algorithm. I tried to use it on a rather large chunk of data and it failed while trying to allocate an array of 32.2 TiB.

One option I can think of would be to cut my data into arbitrary segments of a few seconds each, apply ZapLine, and then piece the segments back together. What do you think?

[marcpabst]

Sorry, I supplied the data in the form n_channels x n_samples, not n_samples x n_channels.

For nremove = 1 it works well, but nremove > 1 always fails:

ValueError: Weights should be the same n_times as X.

I was able to reproduce this error using random data:

data = np.random.rand(300,4)
denoised_data, artifacts = meegkit.dss.dss_line(data, 50., sfreq, nremove=2)

Any ideas why?

EDIT: Removing this part in tspca.py lets ZapLine run (apparently) successfully:

# equalize power of R channels, the equalize power of the R PCs
if R.shape[1] > 1:
    R = normcol(R, wR)
    C, _ = tscov(R, [])
    V, _ = pca(C, thresh=1e-6)
    print("XS", R.shape, V.shape)
    #R = R * V

However, I'm sure "equalize power of R channels" is quite important...

[nbara]

Hi @marcpabst,

Regarding your first message, happy you got it sorted. Let me know if the issue comes back in the future, as there is an easy improvement that could be done, which is to compute the covariances by blocks instead of doing it all in one go inside dss_line()

Thanks for reporting into that n_remove > 1 bug, I will try to look into it this week.

[nbara]

@marcpabst could you please check branch nb/fix-issue-8 and see if it fixes your problem?

[marcpabst]

Thanks for your quick response! The fixed branch seems to work fine. I will report back when I had to test it in more detail.

[marcpabst]

I think it works now fine, although I can't compare it to the original MATLAB implementation atm. Thanks!