Processing EMG data results in null array

[ijhua]

I'm trying to process some EMG data.
It already had some filters (mains + low pass) during recording, so I tried emg_activation() first. But, this resulted in an array filled with nans.
Then, I tried cleaning with emg_clean(). This also resulted in the nan array.
Then, I tried emg_process(), and it resulted in 0s or NaNs in all columns other than EMG_Raw.

I'm really not sure what I'm doing wrong or why these functions don't work with my data.

Thanks in advance!

[zen-juen]

Hi @ijhua, could you try passing your cleaned signal through emg_amplitude() first before passing it through emg_activation()? The latter function would require the amplitude (i.e., the linear envelope of the emg signal) to be calculated first before it can detect the activation related features. Let us know if this works!

So perhaps something like:

emg_cleaned = nk.emg_clean(emg, sampling_rate=1000)
emg_amplitude = nk.emg_amplitude(emg_cleaned)
activity, info = nk.emg_activation(emg_amplitude=emg_amplitude, sampling_rate=1000, method="threshold")

[ijhua]

That also gives me a null array no matter if I clean the data first or not.

[zen-juen]

I see, perhaps you could share with us your script and if possible the emg data (or a sample of it) that you're using so that we can replicate the error and investigate

[ijhua]

Sure. Here is a part of my script. I'm trying to upload my data.

import numpy as np
import neurokit2 as nk

gastroc = np.loadtxt("gastroc.txt")

gastroc

array([-0.00175  ,  0.002625 ,  0.00525  , ...,  0.0056875,  0.0048125,
       -0.0010625])

emg_cleaned = nk.emg_clean(gastroc)

emg_cleaned

array([nan, nan, nan, ..., nan, nan, nan])

emg_amplitude = nk.emg_amplitude(emg_cleaned)

emg_amplitude

array([nan, nan, nan, ..., nan, nan, nan])

activity, info = nk.emg_activation(emg_amplitude=emg_amplitude, sampling_rate=10000, method="threshold")

/home/[name]/anaconda3/lib/python3.8/site-packages/neurokit2/events/events_find.py:92: NeuroKitWarning: No events found. Check your event_channel or adjust 'threshold' or 'keep' arguments.
  warn(

activity

<style scoped> .dataframe tbody tr th:only-of-type { vertical-align: middle; }

.dataframe tbody tr th {
    vertical-align: top;
}

.dataframe thead th {
    text-align: right;
}

</style>

	EMG_Activity	EMG_Onsets	EMG_Offsets
0	0.0	0	0
1	0.0	0	0
2	0.0	0	0
3	0.0	0	0
4	0.0	0	0
...	...	...	...
11976340	0.0	0	0
11976341	0.0	0	0
11976342	0.0	0	0
11976343	0.0	0	0
11976344	0.0	0	0

11976345 rows × 3 columns

nk.emg_amplitude(gastroc)

array([nan, nan, nan, ..., nan, nan, nan])

[ijhua]

gastroc.zip
This is a subset of the original data, but it should give the same results.

[zen-juen]

Thanks for sharing this! What's the sampling rate of your data? Do you also happen to have the raw data but without the default filters implemented during recording?

[DominiqueMakowski]

(@zen-juen we probably should, for all signals, run linear interpolation by default on NaNs values upon cleaning)

[zen-juen]

@DominiqueMakowski yeah, embed them in the _*clean() functions and perhaps throw out a warning when this is done?

[ijhua]

@zen-juen Thanks!

I have a couple of followup questions.

1
We recorded simultaneously from another muscle. Is there a way to do some analysis like cross-correlation between the 2 muscles or some other measure of coordination?

I've tried
nk.signal_synchrony(gastroc,hamstring,method="hilbert")
and
nk.signal_synchrony(gastroc,hamstring,method="correlation",window_size=500)
but both return arrays filled with 1.
Visually inspecting, I don't think they aren't entirely in phase or out of phase, so I'm not sure how to interpret those results. Here's an example of the 2 signals:

2
In the 10s that you extracted, there is 1 contraction of interest. We have another channel that records a square pulse while a stimulus is being applied. I've created events based on that channel, but when I try to use epochs or emg_eventrelated(), it give a DataError.

Here's the most recent try:
signals,info = nk.emg_process(gastroc,sampling_rate=10000)
epochs = nk.epochs_create(signals, events, sampling_rate=10000,epochs_start=-5,epochs_end=2)
nk.epochs_plot(epochs)

It still makes a plot, but I'm confused about the error DataError: No numeric types to aggregate.

And, it's not clear to me how to extract individual periods of activity

[zen-juen]

Regarding your question on cross-correlation, it is indeed weird that you're getting an array full of 1s using nk.signal_synchrony(), unless they are exact replicas of the same signal. We should be expecting values very close to 1 (rather than 1) instead. Perhaps you could provide subsets of the gastroc and hamstring data that you're using so that we can check what's wrong.

The DataError is indicating that the data being merged here is incompatible with each other. It also seems like the EMG onsets and offsets are not properly propagated into the plot (refer to the bottom two empty plots in your figure). Could you print out here what your events and epochs look like?

[ijhua]

I've attached a subset of my data. There should be 3 events in this set. The columns are as follows: Time, Gastroc, Hamstring, Trigger, Comments
sample_EMG.zip
Trigger contains the square pulse (starts about 5s before the stimulus, and ends after the stimulus ends). I used this channel to create events, and the comments channel to label the events. Since the jump always before the end of the square pulse in trials that it happens, I defined events based on the end of the pulse.

The gastroc and hamstring values are not exact replicas, so I don't know why the values are 1. Even if they were replicas, we'd expect the correlation to be 0 because the function calculates phase difference right?

Here are the events and epochs:

events = nk.events_find(event_channel = trigger, threshold_keep = "above",discard_first=1,event_conditions=comments)

Out:
{'onset': array([ 2724338,  3405454,  3969976,  4272486,  4583633,  5003269,
         5396625,  5800708,  6235200,  6687240,  7103214,  7531803,
         7902757,  8331590,  8782189,  9100265, 10203853, 10587904,
        11040867, 11300625, 11557529, 11770170, 11952966]),
 'duration': array([572619, 383065, 208802, 215585, 319194, 294444, 291354, 307359,
        248671, 290804, 311579, 266996, 314603, 335292, 213632, 999989,
        282575, 302451, 160014, 160685, 122703,  97238,  23379]),
 'label': array(['1', '2', '3', '4', '5', '6', '7', '8', '9', '10', '11', '12',
        '13', '14', '15', '16', '17', '18', '19', '20', '21', '22', '23'],
       dtype='<U21'),
 'condition': array(['no jump ', 'jump ', 'jump ', 'jump ', 'jump ', 'jump ', 'jump ',
        'jump ', 'jump - r leg only? ', 'jump ', 'jump ', 'jump ', 'jump ',
        'jump ', 'jump ', 'jump ', 'jump ', 'jump ', 'not good ', 'jump ',
        'jump ', 'jump ', 'jump '], dtype=object)}

signals,info = nk.emg_process(gastroc,sampling_rate=10000)
epochs = nk.epochs_create(signals, events, sampling_rate=10000,epochs_start=-5,epochs_end=2)

The output of epochs is long, so here's the first entry:

Out[]:
{'1':           EMG_Raw  EMG_Clean  EMG_Amplitude  EMG_Activity EMG_Onsets  \
 -5.0000 -0.037937   0.003734       0.000057           0.0          0   
 -4.9999 -0.036312   0.005727       0.000056           0.0          0   
 -4.9998 -0.036187   0.006205       0.000056           0.0          0   
 -4.9997 -0.038375   0.004353       0.000056           0.0          0   
 -4.9996 -0.037812   0.005236       0.000055           0.0          0   
 ...           ...        ...            ...           ...        ...   
  1.9996  0.005125   0.021436       0.000043           0.0          0   
  1.9997  0.006812   0.022485       0.000044           0.0          0   
  1.9998  0.009500   0.024577       0.000044           0.0          0   
  1.9999  0.014063   0.028588       0.000044           0.0          0   
  2.0000  0.018125   0.032145       0.000045           0.0          0   
 
  EMG_Offsets    Index Label Condition  
 -5.0000           0  2674338     1  no jump   
 -4.9999           0  2674339     1  no jump   
 -4.9998           0  2674340     1  no jump   
 -4.9997           0  2674341     1  no jump   
 -4.9996           0  2674342     1  no jump   
 ...             ...      ...   ...       ...  
  1.9996           0  2744333     1  no jump   
  1.9997           0  2744334     1  no jump   
  1.9998           0  2744335     1  no jump   
  1.9999           0  2744336     1  no jump   
  2.0000           0  2744337     1  no jump  
 
 [70000 rows x 9 columns],

[zen-juen]

I tried running signal_synchrony on the data you provided and it seems alright

data = pd.read_csv('sample_EMG.csv').iloc[:, 1:6]
data.columns = ['time', 'gastroc', 'hamstring', 'trigger', 'comments']

data[['gastroc', 'hamstring']] = pd.DataFrame.pad(data[['gastroc', 'hamstring']])

nk.signal_synchrony(data['gastroc'], data['hamstring'], method="hilbert")
Out[175]: 
array([0.86847355, 0.62358962, 0.57553423, ..., 0.86722597, 0.85865494,
       0.98653406])

nk.signal_synchrony(data['gastroc'], data['hamstring'], method="correlation", window_size=500)
Out[176]: 
array([-0.47426556, -0.47426556, -0.47426556, ..., -0.47426556,
       -0.47426556, -0.47426556])

Could you see if this is what you expected?

---

I managed to replicate the error and realized that DataError is due to the EMG_Onsets and EMG_Offsets columns being treated as an object rather than float, like so:

epochs['1']['EMG_Onsets']
Out[178]: 
-5.0000    NaN
-4.9999    NaN
-4.9998    NaN
-4.9997    NaN
-4.9996    NaN

 1.9996      0
 1.9997      0
 1.9998      0
 1.9999      0
 2.0000      0
Name: EMG_Onsets, Length: 70000, dtype: object

And this is problematic for plotting which ideally requires dtype: float64. Thanks for bringing this to our attention! I believe that this dtype: object is probably the case for all our processed signals which have 0 propagated in them and 1 to indicate some specific feature (e.g., EMG onsets and offsets, ECG peaks...). That being said, we'll probably have to make the necessary changes to the internal functions used in *_process() so that this doesn't happen again @DominiqueMakowski 😓

For now, @ijhua you can run this short code to force the columns to be treated as float and plot them again:

for i in epochs:
    epochs[i]['EMG_Onsets'] = pd.to_numeric(epochs[i]['EMG_Onsets'], downcast="float")
    epochs[i]['EMG_Offsets'] = pd.to_numeric(epochs[i]['EMG_Offsets'], downcast="float")

nk.epochs_plot(epochs)

[ijhua]

Ok. I've found out that for signal_synchrony, I need to put in the dataframe column, not the values. i.e. Don't enter df["gastroc"].values just df["gastroc"] Then, I got arrays like you did.

And thanks for the explanation about the DataError!