CNSP: Alright! Can can you hear me?
CNSP: Great Ola? Bialas is a postdoc with Ed Lawlor at the University of Rochester?
CNSP: He's a scientist after my own heart, I'd say, as a grad student, he helped write a toolbox, conducting audits for conducting auditory, spatial experiments and analyzing auditory spatial data.
CNSP: and since joining the lab one of his endeavors, along with one of our grad students, Jendo has been converting the Mtrf. Toolbox into a Python library and leading an unfortunate charge in converting the lab into a python lab.
CNSP: His tutorial today will be full of simulations and interesting minutia, so II hope hope you'll enjoy it as much as I will. I'm looking forward to this talk from Ola, so I'll turn it over to him.
CNSP: No.
yeah, thanks, Aaron, for the nice introduction. And Hello, everyone
CNSP: you're not sharing yet. By the way.
Ole: yeah, yeah, about you.
CNSP: hey? I'm on the screen twice.
Ole: Can you see my screen now.
CNSP: Yep, all good.
Ole: Alright. So I'm I'm very happy that we already got you here to really nice talks about all kinds of different aspects
Ole: of speech and linguistics. Because I don't want to talk about any of this in this talk. Instead, I just want to focus on the basic signal processing aspects of of Trf models. And but before I dive into that.
Ole: the talk is called Fundamentals of Trf analyses in Python. And so I want to take a moment to argue, for why one should use python.
Ole: So for once, and most importantly, it's open source, meaning everyone can access it. It is one off, or even the most popular programming language. And to show that I've included this graph here from a project called Popularity of Programming Language, which quantifies the share of different programming languages over time based on some Google statistics.
Ole: And it's the data is on a log scale. So linear growth is actually exponential growth, and you can see Python is just on a ever steady rise, especially since 2,015, 2,016 triggered by the machine learning and AI boom. It's also easy to learn which is in big parts because it is so popular meaning. There's just tons of materials online.
Ole: And then, finally, it's a general purpose programming language, which means that all things equal, it might be a better idea to learn python than something domain-specific, such as Matlab, because there's just a wider set of use cases for the language
Ole: and for those reasons. And Gin and I wrote an implementation of the Mtrf. Toolbox
Ole: in python called Mtrf. Pi. We just recently published it in the Journal of Open Source software. And so our objectives in making the software were that the Api should be as similar as possible to the Ntrf. Toolbox.
Ole: so that people who already knew that would have a easy time using Mpr. Pi.
Ole: However, we also wanted to use an object oriented design because we felt it. The code will just be easier to use if all the important methods and variables you would be interacting with are packed into one class rather than distributed across many functions.
Ole: We use continuous integration, which means that we have a set of test scripts that runs automatically at any time. We edit the code to to check that. Everything is producing the results it should. And this is just generally a good practice in in software design.
Ole: But it's especially important in our case, because, of course, we want to make sure that running the functions in Mtrf. Toolbox
Ole: and in the end. The corresponding functions in Mtrf. Pi. Produces the same results if you run them with the same parameters, and we ensure that by using this continuous integration process.
Ole: We wanted to keep it lightweight with minimal dependencies. So it right now only uses core python functions and numpy, which is the basic
Python library for doing matrix computation.
Ole: and this also helps us to make the code compatible with all kinds of existing pipelines. And we, for example, did not introduce things, such as custom data types. Because we want to avoid that. People who want to use the code have to rewrite a bunch of their existing pipelines just to fit it in.
Ole: And then, finally, the
Ole: software should be easy to use and learn, which is why we provide a quite extensive online documentation and tutorials.
Ole: So let me give you a quick overview of the code.
Ole: and the package consists of 3 modules. For once the Trf. Class, which is the main class that the user is interacting with. There's also a module called stats, which features functions for cross validation and permutation.
Ole: And then there is a module called matrices, which contains functions to, for example, compute covariances and generate lag matrices, and so on. Those functions are usually not accessed by the user directly, but rather called on by the other 2 modules.
Ole: And then the Trf. Class takes a bunch of arguments, and those are kind of the Meta parameters that define what kind of model you're using. So the direction, whether it's a forward or backward model, the method of regularization or the metric that you're using to evaluate and and compare different models.
Ole: And then the important variables that you're usually interested in are contained in the attributes of the class, such as the weights of the Trf. The time lags and the regularization coefficient. That's 1 1 point where we deviate from the Matlab Api in Python. We can't call it lambda, because Lambda has already defined function in Python which definitely bothers me. But it is what it is, so we just have to call it regularization.
Ole: And here's a little example of how to use it. I use this little function called load sample data, which downloads and returns about 2 min of eg. Recording, along with a 16 band spectrogram of the speech the participant listen to.
Ole: And I create an instance of the Trf class. So here, using empty parentheses, meaning it's just going to use the default parameters which will create a forward model which, with the Ridge regularization
Ole: and then use Tearf train, which is the equivalent of Mtrf train
Ole: fitting the Trf. To the stimulus and response. And then here I use the plot method to get just a quick visualization of this nice spectro temporal response function. Here.
Ole: We also use the opportunity to implement some functions that are not part of the original Mtrf toolbox, for example. Here I use this function called permutation distribution.
Ole: which randomly
Ole: scrambles the stimulus and response trials, and then trains trf models on the on this permuted data
Ole: and returns a correlation coefficient, and repeats this over and over again here a thousand times set by this parameter and permute. And so you're getting a thousand correlation values
Ole: and the distribution of these of these values, the permutation distribution
Ole: reflects the effect size that you're expecting in the absence of any actual causal relationship between stimulus and response. And then you can use this distribution to compare your actually observed correlation to and assess the significance of your result.
Ole: So now I want to move to some demonstrations in the tutorial. I'll be using actual brain recordings. Also, actually, the old man in the C data. So that's just kind of the canonical data set for doing these kinds of analyses.
Ole: apparently. But here I want to just use a simulations.
Ole: because the simulations are a really useful way of testing our intuitions and our understanding of these kinds of models and all the code I'll be using from now on to generate the simulations and the plots I've shared with with the Cns team, so I don't know if you already got it, but if you wanted, you should be able to get it and play around with all of the parameters yourself
Ole: erez agmoni. So I'm using a wavelet here to simulate neural responses, and a wavelet is generated by simply
Ole: multiplying a sinusoid with the Gaussian
Ole: erez agmoni in in time, frequency analysis. We also use wavelets rather complex wavelets, but here I only use a real valued 100
Ole: and the neural response, then, is simulated by con taking a stimulus which is here just some square pulses that are randomly spaced and vary in their length.
Ole: and then we convolve that stimulus with the wavelet, meaning, we slide the wavelet over the stimulus and multiply it with the stimulus at every point, and then for each.
Ole: for each multiplication, we sum all the points and get 1 point of the response. So this basically means we're filtering the stimulus with the wavelet, and the result is our stimulated brain response, which looks like this. So whenever there's a pulse, you get like this one
Ole: mit ctl, and thing here which kind of looks like the wavelet, or like a stretched version of the wavelet. And then, if you have multiple pulses close to each other, the single responses kind of start to blend into each other. So if the distance between the pulses is less than the duration of the wavelet.
Ole: and then we can use the Trf on the simulated response to see if we can recover this wavelet.
Ole: So we're going basically going to act as if the wavelet is not some approximation, but literally is how the how the brain responds to the stimulus. And then we're
Ole: we're checking whether we can use the Trf to recover this ground truth from the data. And in this scenario I'm showing here where there's no noise. There's just the stimulus in the wavelet.
Ole: We can use ordinary least square regression, meaning, we can set the regularization value to 0 and just use Trf train
Ole: to fit the model and then use T. Rf, to predict to get a prediction of the simulated response. And on the left side I'm showing the wavelet and the simulated response. And on the right side I'm showing the Trf reconstruction and the Trf. Prediction. And you can see they're identical. So we're able to perfectly recover the the wavelet in the absence of noise.
Ole: But so now what happens if we start to introduce noise into the picture?
Ole: So if we start adding normally distributed noise of increasing amplitude, then at some point this Trf is starting to break down. So here you can see at minus 30 dB, SNR, it's really hard to still pick out the shape of the of the trf from among the noise
Ole: and minus 30 dB, isn't that noisy? So here in the bottom you can see the the noisy response, and even with the minus 30 dB, just by eye, you're still easily able to to pick out the stimuli from among the noise. But the the ols. Regression is not
Ole: so. We can circumvent that by starting to use regularization. So you can call the Trf dot train function.
Ole: and instead of using a single value for regularization. You can just pass a list of values here. I use a log space array. and then what will happen internally in the Trf function is that it will train a Trf. For each value in this list
Ole: and and test them, using cross-validation, and then select the value that gives the best model.
Ole: the best model, according to the metric you're using, which per default is Pearson's correlation between the predicted and the actual response.
Ole: And then what you're seeing here on the right plot is the optimal value of the regularization coefficient lambda plotted against the signal to noise ratio, and you can see that the the lower the SNR is meaning, the more noise there is, the ha, the higher the value of optimal, the optimal value of lambda. So the more noise, the more
Ole: more regularisation. And with that regularisation we're able to recover the Trf.
Ole: Mit Ctl, and even at much higher noise levels like at minus 40 dB, and here in the bottom you can see the noise response to give you an idea what minus 40 dB. Looks like. So at minus 42
Ole: erez agmoni the response, here you can see the clean response in yellow is really buried in noise, but still we can get back our trf 20.
Ole: Of course it. It looks pretty. Mikey.
Ole: and and that is because we have a lot of high, frequent noise in that signal. Usually in in Eeg we filter out the the high, frequent signal. But here, since it's just plain white noise, there's a lot of high, frequent power
Ole: mit Ctl and one way to to help with. That is to use what we call Tekonov regularization. So Tekonov regularization penalizes the square difference between 2 neighboring weights in the Trf. Meaning it kind of forces the neighboring points to take on similar values.
Ole: and this usually leads to a smoother solution. So here, even at the minus 40 dB, we get a really good approximation of our original wavelet from the Trf, the other other than that, it's the same thing in that. The regular amount of regularization just kind of
Ole: linearly increases on the log scale with with lower SNR,
Ole: but there's also a downside to the Tikonov regularization, which is that if you use it in multivariate models, it can cause leakage between the features, which is something I'm gonna come back to in a few slides.
Ole: So so far we've only used a single predictor meaning. We've used the univariate model. But of course, one really cool thing about these models is that they're able to incorporate multiple predictors and disentangle their relative contributions to the signal.
Ole: So what I did here is I simulated the second stimulus, which is just a sinusoid. and and added a second differently shaped wavelet.
Ole: and then I convolve the the green stimulus with the green wavelet and the yellow stimulus with the yellow wavelet, and II just sum the result of those 2 convolutions to get this stimulus here, which just looks like another sine wave with with with some spikes added on top.
Ole: And then again, in this noise, free scenario, we can use the Trf. With both stimulus features, and we can perfectly recover both of our wavelets, meaning the Trf. Is able to disentangle the 2 contributions, and the weights actually tell you how big the effect of each feature over time is on the signal.
Ole: But so now, what if the stimulus features are confounded? So here I created what in causal analysis is known as a fork?
Ole: So 4 happens when one variable z causes both x and y.
Ole: and that causes, and that makes it so that x and y are correlated erez agmoni. But this correlation does not reflect any causal relationship, and in fact, the correlation disappears once you account for the common cause. Z. One.
Ole: So here z is this sinusoid. and then the another. The other stimulus X
Ole: generated by just setting it to one wherever Z exceeds a certain threshold.
Ole: and it's 0. Otherwise, meaning the value of x completely depends on Z,
Ole: and then the neural response Y here is created by convolving z with a wavelet, meaning X does not affect the neural response. The neural response only depends on z one.
Ole: And so now, what happens if we train 2 Trf models to predict the neural response, one using stimulus feature Z. And one using stimulus feature. X.
Ole: Well, what if if we use Z, we'll just get again our perfect reconstruction of the wavelet. This is just exactly the same scenario I showed in the beginning. If we just ignore X, however, if we only consider stimulus X, we'll still get a somewhat weirdly look in trf.
Ole: but that's definitely not random. And this is because X is actually correlated with the response. It does not, reflect any causal relationship. But you're definitely able to predict
Ole: the response based on X. So if you're just interested in prediction, it's totally fine to use X.
Ole: Now, the interesting thing is that if we then use both Z and X
Ole: in one common feature space, so here I create one stimulus by just stacking those 2 stimulus features, Z and X into one matrix and then use that to train the Trf.
Ole: now we're able to disentangle the contributions. And this is what I'm so now all the weights for X are 0, and this is what I mentioned in the beginning, that the the correlation that's caused by the fork disappears once you account for the common cause. Z.
Ole: But so earlier I mentioned that Tekonov regularization can result in leakage between the features. And now I want to show you what that means. So here, if we use Tikanov regularization, and we start using increasing amounts of regularization. So we increase the value of the regularization parameter from point 0 0 one. Up to one.
Ole: We can see that the more and more we're regularizing with Tikanov
Ole: erez agmoni, the more the Trf. Or X starts to approximate the Trf. Or z whereas here, with lambda equals one, we might be tempted to think that they're both about equally important
Ole: erez agmoni for generating the response, which, of course, is not true. We we know this in this case because we simulated it that way.
Ole: And this is the reason the reason why
Ole: it's usually not a good idea to use Teaconoff for multivariate models, although this effect is not completely absent. If you use Ridge regularization, so you need to. You need to crank up the regularization much more with Ridge to get nonzero values in in X, but it's but it is just technically possible.
Ole: So I would argue that it's always a better idea to just not throw in every feature possible, and let and let the the model decide which features are important and which not, but rather a priori, based on theoretical coni. Considerations decide which features are important. So here the best possible scenario is, if you can already a priori decide that X should not affect the neural response. And so you're just gonna leave it out as a predictor
Ole: and with that I want to thank you for your attention. Again, I would
Ole: mit Ctl and encourage you to check out the online documentation of the project which contains tutorials on all the different aspects of Trf analyses. You can find it at the website called Read the Docs, or just via the projects Github.
Ole: And also, if you're interested in contributing to this pretty new project. Please get in touch both Jin, who you can see here in this picture, and I are very happy to receive any contributions.
Ole: and that's it. Thank you again. And I'm looking forward to seeing some of you in the tutorial
CNSP: cool. That was a lot of great minutia. Thanks, Ola. So if anyone has any questions please feel free to ask.
CNSP: I'll hold my question until others have a chance to ask.
CNSP: But I didn't realize that you had a cat, Ola. It looks like it is not my cat. It's just some random cat in Greece on a holiday that walked like next to me.
CNSP: bummer. I thought it. It looked like Cosmo.
CNSP: Alright. Well, I'm gonna go ahead and ask my question. This is the first time I've actually ever seen the taking off regularization, demonstrating like that that leakage
CNSP: which, thanks for that. I I've always been like aware of it, like superficially and always had like kind of thoughts. And I wonder if I could get your thoughts on it, too? Is it possible, do you think.
CNSP: to implement a way in which Tikanov doesn't leak across channels
CNSP: for
CNSP: other features. Is it in the way that the X design matrix is constructed? Or is it just because there's something next door to it. It's gonna
CNSP: like influence it.
Ole: III really don't have a good explanation of that. Ii think my understanding of linear algebra is insufficient to to do that. But that's by the way, one thing which I think is a great reason to use simulations. If you can't kind of decide it from first principles.
Ole: A good idea is just to try it out in the scenario where you know the brand truth and see what happens. So no, this is actually also just something II played around with to to find out. But I don't have like mathematically grounded answer for you.
Ole: I'm sorry.
CNSP: I just
CNSP: I was gonna see if, like Nick had any, input I just realized that Nick has been a participant this whole time, and not a panelist. Sorry about that, Nick. But if if you have any like
CNSP: thoughts on that like I think Mick's is going to be in
CNSP: in your tutorial. So maybe you could say some words about that there, and we can figure that out because I think that'd be an interesting thing to figure out
CNSP: any other questions.
CNSP: Well.
CNSP: there's a link.
CNSP: Oh, Giovanni, put in the chat. That. I don't know if you knew this, Ola, but we've integrated your Mtrf. Pi into our web browser.
Ole: Oh, that's cool. Yeah.
CNSP: Think it's probably running completely on that right, Joe.
CNSP: Yeah, it's it's all. It's all python, unfortunately.
CNSP: anyway. So for the for the sake of time, we'll we'll cut it there. I will hang around in this in this zoom room, and then some people will be in the tutorial zoom room
CNSP: to answer any questions about setting up
CNSP: your environment for the 2 tutorials. So from here we'll go to the tutorials. Everyone think, Ola, for us people are doing that in the chat. There'll be 2 links that were sent to you. And so you'll just choose one of those links to go to.
CNSP: and we'll meet there in about 15 min. Okay.
CNSP: thank you. To all of the speakers and and to all the participants for interesting questions. We'll see you in a few.