Thanks again.
Alright, can you hear me? Great.
Yep. All of, Beale is a postdoc, with Ed Lawler.
At the University of Rochester. He is a scientist after my own heart, I'd say.
As a grad student, he helped write a toolbox conducting auditory spatial experiments. Spatial data and since joining the lab one of his endeavors along with one of our grad students Jindo has been converting the M.
Terf toolbox into a Python library. And, leading an unfortunate charge and converting the lab into a Python lab.
His tutorial today will be full of, simulations and interesting minutia. So I hope, you'll enjoy it as much as I will.
I'm looking forward to this talk, from Ola. So I'll turn it over to him.
You're not sharing yet by the way.
Yeah, thanks, Aaron, for the nice introduction and hello, everyone. Let me just. Yeah, yeah, about 2.
Hey, I'm on the screen twice.
And you see my screen now?
Yep, all good.
Alright, and so I'm very happy that we already and got to hear 2 really nice talks about all kinds of different aspects.
Of speech and linguistics because I don't want to talk about any of this in this talk.
Instead, I just want to focus on the Basic signal processing aspects of TRF models. And but before I dive into that, The talk is called Fundamentals of TRF Analysis in Python.
And so I wanna take a moment to argue for why one should use Python. So for once and most importantly, it's open source, meaning everyone can access it.
It is one off or even the most popular programming language. And to show that I've included this graph here from a project called.
Popularity of programming language which quantifies the share of different programming languages over time based on some Google statistics.
And it's the data is on the log scale. So linear growth is actually exponential growth. And you can see Python is just on a ever steady rise.
Especially since 2,015, 2,016 triggered by the machine learning and AI boom.
It's also easy to learn which is in big parts because it is so popular, meaning there's just tons of materials online.
And then finally, it's a general purpose programming language, which means that all things equal, it might be a better idea to learn Python than something domain specific such as MATLAB because there's just a wider set of use cases for the language.
And for those reasons, and Jenna and I wrote an implementation of the MTRF toolbox.
In Python called MTRF Pi. We just recently published it in the Journal of Open Source software.
And so our objectives in making the software were that the API should be as similar as possible to the NTRF toolbox.
So that people who already knew that would have a easy time using Mtrf Pi. However, we also wanted to use an object oriented design because we felt it the code will just be easier to use if all the important methods and variables you would be interacting with are packed into one class rather than distributed across many functions.
Any time we edit the code to check that everything is producing the results it should. And this is just generally a good practice in.
Software design, but it's especially important in our case because of course we want to make sure that running the functions in MTRF toolbox and in the and the corresponding functions and MTRF pi produces the same results if you run them with the same parameters and we ensure that by using this continuous integration.
We wanted to keep it lightweight with minimal dependencies. So right now only uses core Python functions and Numpy, which is the basic.
Python library for doing matrix computation. And this also helps us to make the code compatible with all kinds of existing pipelines.
And we, for example, did not introduce things such as custom data types because we want to avoid that people who want to use the code have to rewrite a bunch of their existing pipelines just to fit it in.
And then finally, the. Software should be easy to use and learn, which is why we provide a quite extensive online documentation and tutorials.
So let me give you a quick overview of the code. And the package consists of 3 modules for once the TRF class.
Which is the main class that the user is interacting with. There's also a module called stats, which features functions for cross validation and permutation.
And then there is a module called matrices which contains functions to, for example, compute covariances and generate lag matrices and so on.
Those functions are usually not accessed by the user directly, but rather called called on by the other 2 modules.
And then the TRF class takes a bunch of arguments and those are kind of the meta parameters that define what kind of model you're using.
So the direction, whether it's a forward or backward model, the method of regularization. Or the metric that you're using to evaluate and compare different models.
And then the important variables that you're usually interested in are contained in the attributes of the class such as the weights of the TRF, the time lags and the regularization coefficient.
That's 1 1 point where we deviate from the MATLAB API. In Python, we can't call it lambda because lambda has already defined function in Python, which definitely bothers me, but it is what it is.
So we just have to call it regularization. And here's a little example of how to use it.
I use this little function called load sample data. Which downloads and returns about 2 min of EG recording along with a 16 band spectrogram of the speech the participant listened to.
And I create an instance of the TRF class. So here using empty parentheses, meaning it's just going to use the default parameters.
Which will create a forward model which with a ridge regularization. And then use TRF the train which is the equivalent of MTRF train.
Fitting the TRF to the stimulus and response. And then here I use the plot method to get just a quick visualization of this nice spectral temporal response function here.
We also use the opportunity to implement some functions that are not part of the original MTRF toolbox.
For example, and here I use this function called permutation distribution. Which, randomly scrambles the stimulus and response trials and then trains TRF models on the on this permuted data.
And returns a correlation coefficient and repeats this over and over again here a thousand times set by this parameter and permute.
And so you're getting a thousand correlation values? And the distribution of these of these values, the permutation distribution.
Reflects the effect size that you're expecting in the absence of any actual cause of relationship between stimulus and response.
And then you can use this distribution. To compare your actually observed correlation to and assess the significance of your result.
So now I want to move to some demonstrations in the tutorial. I'll be using actual brain recordings.
Also actually the old man in the C data. So that's just kind of the canonical data set for doing these kinds of analyses.
Apparently, and but here I wanna just use the simulations. Because the simulations are a really useful way.
Of testing our intuitions and our understanding of these kinds of models. And all the code I'll be using from now on to generate the simulations and the plots.
I have shared with with the CNSP team, so I don't know if you already got it but if you wanted you should be able to get it and play around with all of the parameters yourself.
So I'm using a wavelet here to simulate neural responses and a wavelet is generated by simply multiplying assignasoid with the Gaussian.
In time frequency analysis, we also use wavelets rather complex wavelets, but here I only use a real value.
And the neural response then is simulated by taking a stimulus which is here just some square pulses that are randomly spaced and vary in their length.
And then we convolve that stimulus with the wavelet, meaning we slide the wavelet over the stimulus and multiply it with the stimulus at every point.
And then for each. For each multiplication, we sum all the points and get 1 point of the response.
So this basically means we're filtering the stimulus with the wavelet. And the result is our stimulated brain response, which looks like this.
So whenever there is a pulse, you get like this. Thing here, which kind of looks like the wavelet or like a stretched version of the wavelet.
And then if you have multiple pulses close to each other. The single responses kind of start to blend into each other.
So if the distance between the pulses is less than the duration of the wavelet. And then we can use the TRF on the simulated response to see if we can recover this wavelet.
So we're going basically going to act as if the wavelet is not some approximation, but literally is how the how the brain responds to the stimulus and then we're We're checking whether we can use the TRF to recover this.
A ground truth from the data. And in this scenario I'm showing here where there is no noise, there's just a stimulus in the wavelet.
We can use ordinary least square regression, meaning we can set the regularization value to 0 and just use TR.
To fit the model and then use TRF that predict to get a prediction of the simulated response.
And on the left side, I'm showing the wavelet and the simulated response. And on the right side, I'm showing the TRF reconstruction.
And the TRF prediction and you can see they're identical so we're able to perfectly recover the the wavelet in the absence of noise.
But so now what happens if we start to introduce noise into the picture? So if we start adding normally distributed noise of increasing amplitude, then at some point this TRF is starting to break down.
So here you can see at minus 30 dB SNR. It's really hard to still pick out the shape of the of the TRF from among the noise.
And minus 30 dB isn't that noisy. So here in the bottom you can see then the noisy response and even with the minus 30 dB.
Just by eye, you're still easily able to pick out the stimuli from among the noise.
But the OLS regression is not. So we can circumvent that by starting to use regularization.
So you can call the TRF. Dot train function. And instead of using a single value for regularization, you can just pass a list of values.
Here I use a log space array. And then what will happen internally in the TRF function is that it will train a TRF for each value in this list.
And test them using cross validation and then select the value that gives the best model. The best model according to the metric.
You're using, which per default is Pearson's correlation between the predicted and the actual response.
And then what you're seeing here on the right plot is the optimal value of the regularization coefficient lambda.
Plotted against the signal to noise ratio. And you can see that the lower the SNR is, meaning the more noise there is.
The higher the value of optimal the optimal value of lambda. So the more noise is the more more regularization and with that regularization we're able to recover the TRF.
Even at much higher noise levels like at minus 40 B and here in the bottom you can see the noise response to give you an idea what might 40 dB.
Looks like. So at minus 40. The response here you can see the clean response in yellow is really buried in noise but still we can get back our TRF.
Of course, it looks pretty. Spikey and that is because we have a lot of high frequent noise in that signal.
Usually in EEG, we filter out the high frequent signal, but here since it's just plain white noise there's a lot of high frequent power.
And one way to to help with that is to use what we call Tikonov regularization. So Tikonov regularization penalizes the square difference between 2 neighboring weights in the TRF.
Meaning it kind of forces the neighboring points to take on similar values. And this usually leads to a smoother solution.
So here even at the minus 40 dB, we get a really good approximation of our original wavelet from the TRF.
The other other than that it's the same thing in that the regular amount of regularization just kind of linearly increases on the log scale with, with lower SNR.
But there is also a downside to the TCON of regularization, which is that if you use it in multivariate models it can cause leakage between the features.
Which is something I'm gonna come back to in a few slides. So, so far we've only used the single predictor, meaning we've used the univariate model.
But of course, one really cool thing about these models is that they're able to incorporate multiple predictors and disentangle their relative contributions to the signal.
So what I did here is I simulated the second stimulus which is just a sinusoid. And and added a second differently shaped wavelet.
And then I convolve the the green stimulus with the green wavelet and the yellow stimulus with the yellow wavelet.
And I just sum the result of those 2 convolutions to get this stimulus here, which just looks like another sine wave with some spikes added on top.
And then again, in this noise free scenario, we can use the TRF with both stimulus features and we can perfectly recover both of our wavelets, meaning the TRF is able to disentangle the 2 contributions and the weights.
Actually tell you how big the effect of each feature over time is on the signal.
But so now what if the stimulus features are confounded? So here I created what in causal analysis is known as a fork.
So fork happens when one variable z causes both x and y. And that cause and that makes it so that x and y are correlated.
But this correlation does not reflect any causal relationship. And in fact, the correlation disappears.
Once you account for the common cause Z. So here Z is this sinusoid. And then the another the other stimulus X.
Is generated by just setting it to one where ever z exceeds a certain threshold. And it's 0 otherwise, meaning the value of x completely depends on z.
And then the neural response why here is created by convolving z with a wavelet. Meaning X does not affect the neural response.
The neural response only depends on Z. And so now what happens if we train 2 TRF models to predict the neural response?
One using stimulus feature Z and one using stimulus feature X. Well, what if we use z we'll just get again our perfect reconstruction of the wavelet.
This is just exactly the same scenario I showed in the beginning if we just ignore x. However, if we only consider stimulus x, we'll still get a somewhat weirdly look in TRF, but that's definitely not random.
And this is because X is actually correlated with the response. It does not reflect any causal relationship.
But you're definitely able to predict. The response based on X. So if you're just interested in prediction, it's totally fine to use X.
Now, the interesting thing is that if we then use both z and x, in one common feature space.
So here I create. One stimulus by just stacking those 2 stimulus features Z and X into one matrix and then use that to train the TRF.
Now we're able to disentangle the contributions. And this is what I'm, so now all the ways for X are 0.
And this is what I mentioned in the beginning. That the the correlation that's caused by the fork disappears once you account for the common cause Z.
But so earlier I mentioned that Tikonov regularization. Can result in leakage between the features.
And now I want to show you what that means. So here if we use tikon off regularization and we start using increasing amounts of regularization so we increase the value of the regularization parameter.
From point 0 one up to one. We can see that the more and more we're regularizing with Taken off, the more the TRF for X starts to approximate the TRF for Z, whereas here with lambda equals one, we might be tempted to think that they're both about equally important.
For generating the response, which of course is not true. We know this in this case because we simulated it that way.
And this is the reason why. It's usually not a good idea to use TikTok for multivariate models.
Although this effect is not completely absent if you use bridge regularization. So you need to you need to crank up the regularization much more with ridge to get non-zero values in in X, but it's but it is just technically possible.
So I would argue that it's always a better idea to just not throw in every feature possible and let and let the model decide which features are important and which not but rather a priori based on yourtical considerations.
Decide which features are important. So here, the best possible scenario is if you can already a priori decide that X should not affect the neural response and so you just gonna leave it out as a predictor.
And with that, I wanna. Thank you for your retention. Again, I would.
Encourage you to check out the online documentation of the project, which contains tutorials and all the different aspects of TRF analyses.
You can find it at the website called Read the Docs or just via the projects GitHub.
And also if you're interested in contributing to this pretty new project, please get in touch both Jin who you can see here in this picture and I are very happy to receive any contributions.
And that's it. Thank you again and I'm looking forward to seeing some of you in the tutorial.
Cool. That was a lot of great minutia. Thanks, Ola. So if anyone has any questions, please feel free to ask.
I'll hold my question. Until others have a chance to ask.
But I didn't realize that you had a It looks like.
It is not my cat, it's just some random cat in Greece on a holiday that walked like next to me.
On there. I thought it looked like Cosmo.
Alright, well I'm gonna go ahead and ask my question. This is the first time I've actually ever seen the taken off regularization demonstrating.
Like that leakage. Wish thanks for that. I've always been like aware of it like superficially and always had like kind of thoughts and I wonder if I could get your thoughts on it too.
Hmm.
Is it possible? Do you think to implement a way in which Taken Off doesn't. Leak across channels for other features is it in the way that the X design matrix is constructed or is it just because there's something next door to it, it's gonna.
Like influence it.
I really don't have a good explanation of that. I think my understanding of linear algebra is insufficient to to do that but that's by the way one thing which I think is a great reason to use simulations if you can't kind of decide it from first principles.
Yeah.
A good idea is just to try it out in a scenario where you know the brown truth and see what happens.
So, no, this is actually also just something I played around with to to find out but I don't have like a a mathematically grounded answer for you.
I'm sorry.
Hmm. I just, I was gonna see if like Mick had any input I just realized that Nick has been a participant this whole time and not a panelists.
Sorry about that, Nick. But if you have any like thoughts on that. Like I think Mix is gonna be in.
In your tutorial. So maybe you could say some words about that there. And we can. Figure that out, cause I think that'd be an interesting thing to figure out.
Any other questions?
Well, There's a link. Oh, Giovanni, put in the chat, that I don't know if you knew this, Ola, but we've integrated your MTRF pie into our web browser.
Oh, that's cool. Thanks.
Yeah. I think it's probably running completely on that, right, Joe? Yeah, it's all it's all Python, unfortunately.
Oh, neat.
Yes. It is, yeah.
Anyway, so for the For the sake of time, we'll, we'll cut it there.
I will hang around in this. In this Zoom room and then some people will be in the tutorial Zoom room.
To answer any questions about setting up your environment for the 2 tutorials. So from here we'll go to the tutorials.
Everyone think Ola for us people are doing that in the chat. There'll be 2 links that were sent to you and so you'll just choose one of those links to go to.
And we'll meet there in about 15 min. Okay.
Thank you to all of the speakers and. And to all of the participants for interesting questions. We'll see you in a few.