So you need to allow it. You need to. Yeah, log in. And so forth.
And if you did that. Okay. And normally it should be.
You should get this message. And this is important because here's some salts later we will. Call the data.
Oh, I have observed that I still have the wrong, name here. But anyway, you first import the packages and then Yeah.
You need to create, I'll also.
Change. Yeah. So, Did that work for everyone? Hey.
Is the Okay, great.
So then we will visualize the, acoustic speech picture.
And if you run it, you should see something like this. So you still see like some sentences.
Of the spectrogram with added the different center frequencies, which looks nice. I know that interesting speech feature to look at is the acoustic onsets.
The acoustic onsets are The. Half wave rectified. The relationship of the spectrogram.
This actually accentuates all the edges of the spectrogram and it's an important feature because apparently the auditory brain can response.
Hot like or has a more prominent response to those edges than the spectrogram. So I always consider the acoustic home sets or acoustic edges as well.
So like as it's a hands-on material I also included some exercises So the first exercise is to make the acoustic onset feature.
So it's basically the first exercise is to make the acoustic onset feature.
So it's basically you take the derivative of the feature and And to help you, with this.
Check out the, you'll bring, documentation because the whole, Workshop will rely on the 2 books, ill brain.
And this spectrogram is already an and d variable. So and and you'll bring object, so to speak, which has methods to take the derivative.
And to clip, the negative parts to 0.
I will give you some time. Meanwhile, if there is any any question or if you want to give any feedback on where you're at in the workshop then.
Please. Let me know just either unmute or put it in the chats. And then in a couple like in a, yeah, few minutes then, we can.
Okay. No.
Yes, installing ill brain takes quite some time. That's also like If you ever, want to install in, use, Mumbai is really a time saver,
Because they can using, or whatever it takes, quite a long time to
Do. These methods work in a similar way to Matlab in that if you have something with I'm assuming the Spectrogram channels are organize along columns?
Does it? Do the thing along columns.
Maybe it's in there and I didn't see.
Well, if you go to, Okay.
Yeah.
Oh, there it is.
And, and you, for example, want to take the derivative. Then you can really save the dimension and the dimension is really It's super straightforward time, frequency, and it's Yeah, in this case you want to use time.
As a spoiler.
You want to use what?
Oh, I see. Yeah.
You need to take the derivative over time. Then you will have so
Okay, yeah, let's. I'll, I'll make this first exercise so you Oh, I see there's an, yes, I, sorry, for the mountain stuff, I last minute yesterday we named the CNSP underscore workshop so in the This one data routes, so you should rename it to.
This if you're replace it the CSB worship it's to check one then it should solve.
Brian, can you let me know if it worked out? And then we're.
So back to the code. So if you want to, . So we go It's when we make a new variable, interested onsets, we use a spectrogram and then we take the derivative over time.
Next, you can also use the clip function and it's, you can also use the clip function and it's, you basically, it's like to pick, to click at zoom so all negative parts will be kits to 0.
You can run this and then add another, so and I can, for example,
I'm going to take the same.
So the sub function is basically.
Telling. Them to take, yeah, only like a very small part of the data because otherwise, you know, it's a We need to get the somewhere.
Now you see it's way more, sparser than the spectrogram, which is normal because like the If the spectral ground would be like this, the edges would be more like this.
So yeah, in a in a next exercise I quickly wanted to show you how the.
Which apparently I already, to answer. So, to show you the relationship between the spectrogram and the edges.
So if you just And, yeah, uncommon the first line. So either you do, you can take the, over across a frequency like this.
So.
Okay.
Okay. So if you run this, then normally you can see the relationship between 2. So, Brian, sorry.
So you changed the data Like this.
And from then on it should. It should run normally. Does it work out for the other participants?
Yeah, that worked for me.
I hope it's working out for you guys.
It worked for me.
Okay, you might need to respecify other parts of the redirectory name depending on where you saved it in your drive.
Yeah.
So like I didn't save it in my drive. I saved it in a sub folder of my drive called Google Calabs because then it when I added shortcut I needed to put it in another folder.
If you use that.
Okay. Yeah, thanks. That makes sense. Good.
Does that? Fix the trick.
Anyway, if it's not working out, I will share. Like this will be available the Google Drive will still be available until Kailover kicks me out.
And all the quotes is on, my, so. You will have the resources.
To replicate this.
Okay, so. Then. This is about the audio feature.
So basically we have the spectrogram, I appreciate onsets. That's sufficient. And we went already to the code so we can move on.
And next up are just flexible features. So basically we want to to make this sparse array of features where there's a spike at every onset of a forum or or word on sex.
So basically what do we need? To do so, you need to have a transcript so you need to know the words which were spoken during the audio fragments.
And also you need to have a text script file or like it shouldn't be necessarily a textbook file.
And it can also be another file, but basically it tells you when exactly a phone in was spoken.
So, first maybe, a bit more on Visper Kicks. So, at Yes, this link.
We, basically what you just should do, but we don't have time to go for it right now, but I give you all the tools.
Just install this repository. And it's very, very nice. And then, the install installation instructions are quite straightforward and well explained so normally you should be able to manage.
And then basically what I do is just running this line. In in command line where in my case this is NO because I'm using Netherlands Dutch.
And the name of, and then it will outputs you like this and in the end really a text file with Hold the lines which has have been spoken.
Which is really really nice. I noticed only for the last 3 months or so. So the majority of my PhD have been transcribing a lot of stimuli.
So this will save you a crazy amount of time, time. So that's really, really cool.
The thing is, it's super nice, but there are still sticks in it, so just make sure you also do a manual correction afterwards.
Then, secondly is to get that phone in segmentation. And that's actually also not super difficult.
There's a like in my stories we all we always used our own K Lover stuff but This, bus, web service is also super, So basically, what you do is and you includes the The text, the output of the B.
And you also includes the WAF file, you open them, you can upload them. And then basically, after after it's uploaded and you changed the, the language to Dutch, which is, the language of the buffer fragments.
You can run this and it will output you a. It will output you the Ti, the text read file.
This like, yeah, I am not going to do this right now because this again, takes a couple of minutes, but now we have the tools to make your own text, files and you see it's not rocket science.
It's quite nice and easy. Then again, If you want to go for it, then I also suggest you use I'm quite convinced.
A lot of lots of people. No, that already. But to make sure that's opens.
A window like this where you can really, Thank you. Open the text and the, .
So the text click on the Thank you. Really open them together and you can really play around with all of this.
And you see in the beginning, Maybe I'm quickly going to check whether audio responses shared.
Okay, I'm not sure, Do. Can you hear this?
No. Anyways. Here, and what I'm hearing is music. So here the the segmentation is not on rights.
It's it becomes it becomes good. And so just. Make sure you check your, segmentations, your transcripts, make sure to check them afterwards for music I would suggest to for example with Audacity, cut out to music to make sure your foring segmentation is downwards.
So as you see, making, making a transcript near a text switch file. It's not rocket science, so you can do it too.
Okay, through the code then.
Okay.
Hey, Malis, before you go on, there is a question in the chat. So it's asking about acoustic concept.
Is there a recommended paper investigating the relation of the acoustic concepts with actual linguistic elements.
The the paper that I would, recommend is from, sorry, brought back.
It's not necessarily related to linguistics because linguistics is still a bit of a newer field.
But the Brokeback paper is super interesting because there they compared, how, how you listen to an attendant and an Nick North speaker and what they see is that whenever an attend speaker is masked by another like an ignored speaker or distracted by a destructor, then you can actually see that the acoustic.
Masks on sets get unmasked in case they are attended to but not in case they are in I hope this answers a bit of the question.
I think There like I just want to highlight the importance. I think including acoustic concepts is really important.
Because of this feature, but I don't know the exact. Variation with the linguistics because in my or on the top of my head I don't know a paper that's directly investigated this.
Yeah, okay, great. Thanks. I would, I know the paper and, Like I know that it is a commonly used feature, but I just actually curious like.
As with the many other features that we often use, the stone correspondence. Like linguistic elements but like the like, yeah, I would like to know.
Thanks for your question.
Okay, thanks.
Marley's 2 months scrolling up for just a second to get the end of that. Mine?
Yeah.
I am. Yeah, sorry. This is basically the uncommended, like it was in a common, so you should just remove a hashtag.
Oh. Sorry, keep going.
So yeah, then there's a really nice, toolbox. Which basically allows you to reading this text so although I already made it like as an ill-brain, or I made already all features for you, but maybe it might be nice to make an D variable ourselves.
So basically we're first going to load and the text so you have the Here's.
Where the text is saved, the parts and the sampling frequency of the then what you see whenever you load the text scripts, using the, yeah, the text then you will see that you basically get an output of.
Yeah, a dictionary with 3 keys. So you have the automot, you have the come out and you have the mall.
And in the ortho you, so you have the words. Any so for example if you do like this you have a lot of intervals and each interval still tells you which words has been said.
And when the onset of that word was and when the
So, Okay. So to extract the word on sets time, basically you loop over all these intervals in this order, mouse, and elements.
And you just take, Sometimes there are empty words. Really empty. So those, you discards because those are those are silences and those you don't want to get.
Then how it will bring works is you need to couple a lot of, or you first need to create a dimension before you can create a feature.
So in this case we're creating our time. Dimension. Which is.
From 0 to the amount of samples that we require and then the sampling rates that the feature should be.
And then next stop recreates the instance. And from all zeros, so we make a feature with all zeros and then at the timing of the word on sets we just put a one.
And then you can of course visualize this. So basically what I visualize here is Yes, the spectrogram, which is averaged across frequency bands, hence it's only one band.
And then I also visualize the word on says features or hearing blues. So whenever there's a small peak here, there's
Okay, so next up as a next hand on hands on exercise, I challenge you to make, phoning onsets.
It's very, very similar to word on sets, but you should take the. Instance and I I give you already a hint that's unknown utterances are given by these elements in the Yeah, So I'll give you, some time to, to make this exercise.
And meanwhile, if there are any questions or issues still, then please let me know. Also, Brian, can you give me an update on whether,
Whether it worked out.
When you, I have a question about the phone name segmentation. The actual segmentation through mouse.
And the manual changes. How how exact would you say do you do this segmentation because we found that when we were trying to manually correct it sometimes like auditorially it's hard.
To find the exact boundary of the phonemes. And we do it, we separate between words, syllables and phonemes.
So we try to use the other boundaries to help us, do you have suggestions about how maybe to improve on that or is that just something that sort of rely on.
More like less exact.
Yes, that's English very tricky because some, phone names they start rather silence like foot or I don't know whether this is a correct example but like like, you know, when you say hello, like.
There's no super hard edge at the beginning. That's indeed tricky. I.
I usually do this together with colleagues who are audiologists and which are who are really trained.
And in doing stuff like this. Yeah, it's also something that you learn to practice.
It might be Good. Might be nice maybe to have. A good reference system to make it more uniform because yeah I rely on how colleagues taught me to and to do it a lot with colleagues who have way more knowledge.
And I have because I'm not really trained to use. I know it's a it doesn't 100% answer your question.
No, it's good to know that everyone is in the same, is in the same realm. Like everyone's, tackling the same, is in the same realm.
Like everyone's, tackling the same challenges.
Yeah. Yeah, hoping that you have a good, for in segmentation, tool like the bus services or so.
So we don't, we can't use the mouse, because we are, Hebrew isn't on the, as one is not one of the languages.
That it offers so we needed to find different. Different methods.
Yeah. Yeah, previously we also, used in-house. A thing a guy created.
A couple of years ago but Yeah, it's It's always tricky when you are not English speaking.
I feel, hopefully, . The bus services have, they do suggest something. They do have some suggestion on how to deal with language languages.
They don't recognize but I never checked that out so I don't know whether it's wording to spend time.
Okay, let's make this exercise so basically I'm Just going to use.
S.
Bye.
So, basically, once again, Let's see.
In this one this time we only want to know instance Okay.
And we don't want this to be at like one of
Right.
Also having problems with copy and paste.
Yeah, weird. Normally it's not such a pain. Yes.
I don't like it.
Okay.
For a bit bigger.
Sentx.
Okay. And then we can create. Okay, let's see. Yeah.
Okay.
Yeah.
So, Okay.
One maybe.
Yeah, where does this? Hmm.
Let me see.
You know, and maybe I'll immediately multiply them by 2 so you can see.
First, Okay.
If you put another record in there all, on the same
Okay. So. Then you see. The phoning onsets which are slightly higher.
Okay, cool.
Nice. I hope that worked out for everyone.
So yeah, we created now just the acoustic features already, the seplexical features.
So let's go quickly over the linguistic features. So basically we want to know in person with the amount of information.
For each phone. So first a very conceptual, description of what is funny in feature features are before we're going to go really into the yeah we're not really going to delve into the mathematics so that's might be good so but imagine whenever you hear this on bar there are a lot of words starting as well with a bar and the idea is that our brain activates all the active
words. And based upon these active words you can actually calculate for example the volume surprises and so how likely the is in this whole award of words.
Or you can calculate the cohort entropy. And this is actually the Shannon entropy.
So basically what this formula is. Of these of the compatible. And Yeah, words in cohorts.
It's basically the degree of competition. You should see this if there are a lot of likely words.
If there is, if in the words of activated words are lots of words which have a high Word probability.
Then the degree of competition of which work it's actually going to be is high. But if the court if there is only one item which has a high probability and all the other items are fairly low.
Then of course, you have a low degree of competition and your co-host entropy is low.
The general trend of these foreign features is in the beginning the first phonemes in a words have higher phoning surprises and cohort entropy.
And the more at towards the end of the word, the smaller the value becomes. That's the general trend of how it works.
Because then you have a better prediction on what the word is actually going to be. For example, if I tell L of then you are probably aware that the word is going to be elephants and the last don't really matter that much.
I, in my studies, I always computed it using just the word probabilities of the standalone words.
So independent of the context of the previous word, so the code that I share with you is also using what probabilities of the so in order to create these linguistic features, we of course need a pronunciation.
Dictionary. And a pronunciation dictionary is a mapping of a word and how the words would be pronounced.
So to to create this to create a. Pronunciation dictionary. We you again have this Bustable and there is the G to P and basically you just input.
Inputs a file with all your words that you that where you want a phoning segmentation from and in included here.
So based upon my previous studies. I always use the word frequencies of striptex. I think I also have a slide on this.
I have here. So I use the word probabilities on Sotalex. This is also available in lots of languages.
Not sure whether it's available in in Hebrew, but anyway, and any any word frequency measure that you have will work for this.
You just. Yeah, need to extract it another way. So basically what I've done in in my code is first to take the words from Subtex.
This is a very like stupid script that it's reads in Subtlex and it just takes all the words and yeah, we move some of which have like are very infrequent because for often they are very weak words that I've never heard of or have very weird symbols in it and just opinion it up a bit.
And then There is. A maximum amount of words per file that you can insert in the bus web application.
So then I just write multiple files and then basically I have,
I, get. This subtext words and that's then the file which I insert in this in this bus tool.
And then after a couple of minutes. It's, it gives me this, where you have a words and then the.
So, yeah, so based upon this, you can basically go ahead and, compute, words, and go out to entropy.
If you don't want to do, all the meta, where I have I wrote these functions which you can find in the GitHub repository.
And that you can use to. Calculates your own phoneing logistics. But for now, yeah, I don't know.
But from now, for this workshop, I just provide you the outcomes of these models because also creating this.
Like phoneme calculations it for example it takes me like it takes almost a full night before this 15 min long audio book is fully that I have the phoning in physics for this audio book.
So it takes a while, so it's not possible to do this, during this workshop.
Good. Now you have to coach. That at work level, so this is the example, from previous, so.
At what level, I use, work surprises, so basically it's a probability of a word given the 4 previous words and to do to do so I use a simple engram model unfortunately I haven't found a way to share it with you yet because it's too big a bit and it's a bit too big to share but if you're interested in the Dutch version then just send me a
message. The word probability is just, yeah, the probability of the words. So as I said, I use.
This is yeah, a fairly simple model and to create this I use the L LM toolkits.
Where you can write, yeah, the guitar page is pretty yeah extensive on how you should train a language model and it results or gives us an output model which you which you will use to estimate the word probabilities.
So in the in the file. Nice. Here I think.
So you loads, the output model and then you can really using this Apple toolbox. You can just get, the work probability with this very simple, line of codes or the probability of a word given, the previous words.
But those are already created because it takes a while to compute these features. But we can look at them and go and Okay.
So, here I. Show you the. You're right from, And then I challenge you to just.
Loading all the other features. So you can see how they they are in names if you open stimuli, then you have all the features.
Basically just. Writing a row of Oh.
Okay, can we name it too so you can, then I'm not going to be able to paste.
Okay.
My face hards.
Everyone having complete based issues? Because that's getting
I'm not able to right-click copy and paste, but I can. Use the control C and V on the PC keyboard just fine.
Oh, nice. Cause that doesn't work either for me.
Thank you.
I think it's worth surprising and gram.
Yes.
Sorry if I'm keeping you guys.
Hmm. Let's see.
You chose the same thing for though. The word frequency, word.
Okay.
So I'm gonna add, offset to each.
Before so we can see.
This is, if I, thought, of course.
And then you see.
Okay.
. Which So now we have. Features.
Then I went to just quickly show the slides like before. So basically now to assess the ed value.
We just need to create 2 models like a complete model and a baseline model where one has all features.
And one, where we have all the features except for the linguistic ones and then we take the difference between the 2.
As a quick. Notes, the ill-brain toolbox uses boosting. So this is not the same as we saw in so it doesn't use, reach regression.
To quickly explain how it works. So basically it fits, it estimates the TRF first by flat line and then it's basically at each sample of the TRF.
Dependent of like at each sample this TRF, it starts predicting the EG using a small increments and size and it sees how well the model performs then it does it with the next sample it puts a small increments on it.
It sees how good the ball performs and that that it does for all possible samples. Positive increments as negative increments and it then it detects which increments on which sample the model performed best.
Then it puts the small increments on that spots and basically then the same process happens with this as a baseline.
So basically, then a new increment is added and fitted on top of all. Samples positive and negative to see where yeah this increment is best placed on top of this one or maybe here a negative one.
So it's a different procedure. It's an iterative procedure. So just to Keep in mind that it's slightly different.
Then what all it explains. So she's, first I, I just take one subject.
The EEG of one subject and I make an D variable, from. Yeah.
From the EG and to do so you need also a sensors dimension which I created here so that it's that Ilbrain knows that it's a sensor dimension that it has a sensor dimension and a time dimension.
Then, and you estimate the, model. By calling boosting. Here I limited the EEG to, I just use 3 min of data.
Because yeah fitting a model takes some time. I start from 0 ms and I stop at 500 ms.
So 0 point 5 s. This is the basis function, the basis function is basically the kernel that you convolve, the TRF.
So basically you should see this as these are all very small spikes. And the basis is like a kernel that you convolve with these windows so all those spikes become becomes a little bit
But yeah, so that's run this this should take. Not super long, but takes a bit.
I hope that's, working. For everyone
What is that basis? Value. Represent. Standard deviation, something like that.
Oh, okay.
Cool. Thanks.
It's, it's also a time that the, of this, it's also a time that the, of this, yeah, of this kernel, which, but there's a small exercise here where we will.
Play with the width of the basis and then you will clearly see how it affects the TRF.
The basis function like it's a definition of the kernel that you convolve this Pikettyreflets.
And if you want to go into more ill-bringing stuff, you can also say which kind of window this should be, whether they should be a hemming window, which is his default, or a Gaussian or whatever.
So. Yeah. Does that completely answer your question?
Yeah, it does. Thanks.
Is there something that you do when you input the different features? To account for the fact that the acoustic envelope is at a different scale.
Like do you scale the features the different features to the same?
Yeah.
We're seeing, normalizes the features.
They don't do this outside like the this, the toolbox does it automatically.
Yeah, you will see, yeah, later on or if you are playing with the, with the results and you have an H and an H scales and contrary intuitive the age is the scaled one so when everyone everything is normalized and aged scale is if I'm not mistaken, So if you want to.
That's the effect of. So I hope the boosting models ran for everyone. And so in you can visualize the, prediction accuracy.
Yeah, by running the next cell and the TRF. So those 2 important outcomes of the forward model.
Like this. So this is just, one subject. Fairly high. Prediction accuracy.
Which is probably just due to and then the visualization of the TRF where you see this peak is around 200 and like a peak around 250 s.
So then we're coming to the exercise. To. Play around with the basis version.
Sorry, I don't know if I missed it, but what did we include in this model?
And this is really just running one forward model, so we have here. The spectrum. So this is the EEG and Spectrum.
Yes.
And we start at 0, until 500 ms. That's the window of, from 0 to 500 ms.
This is partition, 5 partitions, so meaning. That we divide our data in 5 equal parts.
Where you use here one partition for testing model and 4 for estimating the
And your data set you said is just a 45 min. Audio.
Yes, but we, but to hear, like here, we just load one subject and we just take the first 3 min.
Okay, so those 3 min are divided into 5.
Yes. I will, I can hear, plan to show a bit more details on the.
On how do you say partitioning works. So.
Okay. So, we have the, prediction accuracy in a TRF.
So let's play with this basis function.
For this I will just limits the That's the time for training now until one minutes because taking like using only 3 min sticks already quite a long.
Okay.
So. Okay.
Here, I, I will just. If it's the start and stop any amount,
Okay. This.
Yeah, I will disable the testing partition just so You should. Just so it's.
It takes less time. Okay.
Okay.
Yeah.
We need to take a
Something.
Because I didn't improvement places. Sorry. This is a stupid.
Okay. Is everyone still a bit? Following ish
Okay. And I see that for longer, Colonel, maybe I can.
. The The basis means.
Yeah, let's walk through this because of the voice. I lose time again with, for the long basis.
We know you see that the TRF is very, very smooth and depending on just like the shorter your basis kernel is.
The yeah the more sparse and I generally use, 50 ms basis, function, but this of course depends also on your sampling rates.
I generally, use, 50 ms for, hundreds.
On the 28, sampling rates, but for 60, for It's probably better to check out.
A different basis.
Yes, so the Colonel simply, yes.
Like in this in these cases it will be be just nearly an impulse. If you go broader then it's more like yeah in this case it's a.
If you go broader then it's more like yeah in this case it's a having window but think of it as a bit of Gajan, but it's all the Gaussian that it's just smooth salads, all, all small.
Okay. Yeah, so basically, I would like to challenge you to, make.
Basically the same, model, but then Next to the spectrum also includes phoneing offsets.
And then we can investigate the added value of, You. So we can see 3 min.
And this time, so then you, included as an, Is there any questions meanwhile then, please, Let me know.
Okay.
Yeah, you need to check out how I
6.
Okay. Okay.
So there is a discussion in the chat about what is, this buses kernel exactly?
Yep.
So And then. Wait, so Andre is asking, so the business parameter specifies the width of a humming window specifies the width of a humming window which is the
Yes. Yes.
Yes, and then you do this convolution. So this multiplication over time to have your sticky TRF into something way more smoothish.
Okay. Thank you.
And then, the same specifications. Which was. Okay.
I see for what reason would, sometimes it performs slightly better when it's suited.
Because for example, you will have due to training big like lots of peaks like this and due to smooting your your peak will look like this.
And that might slightly improve the testing on the testing partition. Additionally, why you want to also smooth a bit is just to, to allow it to be more generalizable across all.
Participants. Because if you work for example with a very sparse For example, let's say that it's for subject one like this and for subject 2 it's exactly the same with opposites then the average yeah okay maybe a better example maybe should should be a small peak straight next to it then at the average TRF would look I have a, would look like like this.
Well, in case you would have smoothed it, smoothly, both peaks a bit broader than it would be maybe like this.
Is that, Kish?
I hope okay.
I think I use, Okay.
Yes.
Go ahead.
I have a small question. Here in this line that you're Doing this model.
It says time equals 0 to 180 and on the one before it says none. To 1 80 is there.
Yeah. Sorry, I'm quickly checking. Whether my arguments are, yeah.
I don't, Where does it say.
Here's this one.
Here in the one they were looking at before here. No, down more. In the boosting model, yeah, here.
Here's, because the first sample, will be, so then, because this, so takes basically a subsection, from a point up to, 180 s and others.
It's known meaning saying like just take up to this time. For example, if I would have, for example, if I had a TRF from minus 100 to 300 ms and I would say time going from non to 0 point 3 then I would get from minus 100 to 200 ms if I would do 0 then I would only get the 0 to 300 ms but now
In this case, because it's the same.
yeah EEG and spectrogram both start at 0 so it will be the same
Okay.
It's more like the first sample of whatever you're indexing there.
Yeah.
Hmm. Cool.
You read this. Should have said. That's.
It might be the browser. Or something about, I don't know what browser you're using, but something says it doesn't.
. So sorry
Yeah, it's weird because previously it worked out, but now it's acting. Weird anyway, doesn't know.
As long as you pair with me and me tripling and typing, that's okay.
Thank you. You can see like a frustrated coders in front of Okay.
What is the end of that code? Say the
So basically I against the basis window, the amount of partition and selective stopping. Selective stopping is actually so when you have this algorithm It's iteratively so it will basically never stops unless you tell it to stop.
And this says like take the stopping, criterion L one, so it's that the Eros, should be based upon the, oh, I'm sorry.
I'm missing up with different arguments. This is saying that after that one, iterative step increases in.
In errors so the prediction becomes worse then you should stop with the fitting of the TRF.
Make sense.
About the L 2 and L one stuff you can also have an you also have an argument at all so I was ethics.
Yeah. Caught up, or, so you can block the, Prediction accuracies of this R 2 model and harder TRF of the spectrogram looks and how the TRF of the pointing concept looks.
By the way, these effects here as I said, these are edge effects. And these generally abort, and whenever your TRF is trying to explain something which is happening outside the boundaries of your integration.
And that can happen. I would suggest to increase. The.
The integration window here. Or sometimes with adding multiple features these effects and disappears.
I would really suggest to, in like increase your integration window, so the T start and T stop.
Especially when these effects are prominent on group level subjective specifically it might be still a bit noisy and stuff but Yeah.
So here you, clocks the top of butterfly and you can, set the timing of this topography using the So then, let's, together determine the added value, of.
Okay. Onsets in this case before we do the or throw on to the next step.
Yep.
I have a question. Our plots are very similar but slightly different. Is there like some randomness built into the boosting algorithm?
Like does it like sample?
Yeah, well, it's normally it should be a fairly similar.
And you use it.
Oh, it is. It's fairly similar. It's just there's, you know, some really subtle differences and I'm just curious if there's like any like optimization, randomness built into it.
I just don't know how the boosting algorithm works.
Yeah, not that I'm really aware of unless there should there is chain or that something has changed.
And, that's a bit weird too.
I mean, if it was just choosing at random, say, let's put an impulse or let's put a kernel here does that help no and then randomly
Yeah, but it's still, then decides which one is the best one and that should be in my case and in your case exactly the same.
What if, but if it starts off with, you know, maybe the first couple of steps that are slightly different and they all like add some value, at least at the beginning things can be adding some value.
Yeah. Yeah.
Maybe like it goes up on slightly a different tree. I mean, it's very similar. I just like, for example, that the low V at the far left.
Has a few other like lines that are close by and instead of the big gap. But I mean, like otherwise it's like almost exactly the same.
Yeah, I'm I'm not completely sure. Because for example how large the this this The speakers said and so on.
Yep.
Hmm.
Those are all defaults normally, but probably if it's, if it's slightly different than probably there should be some small run inness in the code, but I'm not sure where it is introduced.
Okay.
Like on the top of my head. I didn't think so, but maybe I miss something more recent versions or so.
I know.
No worries, maybe we'll ask Christian.
Do you send
Yes.
So that we can take the difference, and then we can, visualize, the topography.
Using the Okay. Per month. And
Yeah. And then, yeah, these are just some arguments. And to make it But I like Zoom.
Nice. Thank you.
We can also put the, additional. Pull bar. and then you can see.
This one.
Yeah. Well, this really high for a difference in projection.
Let me check.
Okay.
But probably it's just because the values here are so crazy. But it because it's situated more sears.
You can set the. It works. We're ready at values.
Okay. So that's basically how to calculate the, value here. I just wanted to give you additional like just to yeah, have a happy.
Have some feeling with this. Segmentation stuff. So in this case we use 5 partitions.
And if you don't use the test argument, so if you do test equals 0, then you see That's, you'll bring basically before it starts testing model divides, your data in 5.
And uses 4, partitions. To train the model on and this is for validation so to determine when the fitting needs to be stopped and to apply.
Yeah, validation, of the model and to determine, Yeah, when the fitting of this implement switch slop, if you use a testing partition.
Like this.
Then you see that there are way more splits and that it also takes way more time. So then you feed a model on 3 partitions use one for validation.
And then apply the model on, on this phone. If you load the TRS, so the the R dot H then you.
Obtained the average of both kids. Okay. Is that okay for everyone?
Is everyone still clicked on the same check?
Would you scroll up a little bit to where you did the difference in the prediction accuracy?
And this part.
Yeah. So. So what we do here is we.
But R 2 was the Oh.
I, yeah, our 2 is the model we trained, the last, including the 4 months.
So it was the combined one of the phoneme and the
And then we subtracted the spectrogram so we have the added value of the, well, what do I do here?
Hmm.
We have the edit value of the, Okay. So. Will there be answers to the exercises?
Yeah, sure. So I will save this file and then whenever you open the whenever you open the document again, you will have the 5 with all answers.
Exercises. So no worries. And also I will give you a link to the GitHub repository so you can.
So you have all the code. So, and now, I want to show how the linguistic features actually look like.
So basically we have all the tools. We have all the features. We know how to estimate a TRF to get compute the edit value that we just did for phoning onsets.
Now we use exactly the same pipeline, pipeline, but use it for the invested features. Because this takes a crazy amount of time.
Because you do this for 15 min lots of subjects. And yeah, you have lots of features in a model of this takes quite some time.
And so I already, . Make sure you don't
Okay, so I will show you I pre trained all the TRs already for all subjects so for every subject you have a complete model and you also have a baseline model.
So basically we will. Just loading the base their complete and baseline model for every subject and we'll do the difference between them.
Noes. Yeah, this is exactly the same outcome as you get got in the our instance here.
Okay. So basically this calls the TRF, directory that I just show you with all the pre-trained TRS.
And then it's loads it and then puts it in a in a row to make an ill-brain data frame so If you showed this instance.
Yeah, it has the subject name, the model type, whether it's complete or baseline, and then it also has a, an ND variable, called prediction person.
So then using Top Map, you can really, easily use this, data frame to show.
In the prediction accuracy for each. Model.
So then you see on the first side, you don't see a clear, difference between the 2 models.
Then using statistics, which is like, really, handy to, use the, toolbox for.
So if you do that. related. It's not like actually it's a permutation test.
Based on that, is that also felt that will be applied to test whether a prediction accuracy, is significantly different between the 2 models and it.
It says like use the matching factor subjects. So, if you,
Sorry.
Name Na. So that, A.
Sorry about that.
Okay, so. Then you you can see the difference between the 2 models. So here in in the, like we know there's a significant difference because there are these curves but they're the color part is way too there's a significant difference because there are these curves but the color bar is way too, to, to actually visualize the difference because the added value is small from
linguistics. So if you do rest, mass difference, you can actually plot it better.
And then you see in blue because it's baseline minus complete and the complete model has the added value of the linguistics.
To So, the computer is higher as baseline. So the, the difference should be negative.
And then you see that there's a problems in improvement here in central areas of the brain where we expect that linguistic features.
Have, an added value.
So,
Yeah, to maybe it's it's not yeah, super interesting because we know that there is a significant difference between the 2 but just for sake of.
Exercise we will, make a new data with the difference. Then.
No, we can plot its, rights. So it's nice, I guess.
So we loop over all the subjects.
Let me take.
One.
Well, working with, I'm sorry. If you're typing exactly the same as I am doing,
So we want the subject to be, the same and then the, should be. Thanks.
And then.
Who is this? And then I like to use a lot of, functions to make sure that It's the right.
That's how I expect it to be so I expect only one correct entry corresponding. Do you this line of quotes.
Okay. One Yes.
This time we want No, that's fine.
Baseline level.
I'm not. I'm sincere. So the video errors make their votes on mute.
Let me check out.
So we have our rows. And we need to make it into the data. Okay.
Okay, and let's. Let me show them.
Okay.
Okay, so this is, what you expect. It's very similar to what I've shown about.
So if you want to make it more beautiful, we can. Okay. Which I like. Okay, beautiful.
Yeah. Last minutes, I hope I didn't include an exercise anyway. Yeah, I did still. I'm sorry.
Maybe I made it during. Anyway this is to load the TRs we can also just quickly go over it.
Basically, loads again the TRF. I only look at the complete model. Because that's like what we're interested in.
In if the so You'll bring, in a data frame. That the ND variables saved in the same columns would you say to have the same dimensions and because the spectrogram have the frequency dimensions and the for example the GRS for word surprise or don't I take the average across frequency to make a nice data frame
I have.
Yeah, here again, the, TRF of the spectrum. Yeah, how to use this.
Channel selection to take, the mean of.
A certain channel selection, so this channel selection we defined earlier. It's this here. So.
So these channels and then you see that if you have. Like lots of participants that you have really a very beautiful, TRF.
Nice. This is also a nice function if you want to, do a closer based permutation tests.
Okay, because, to see where the TRF is significantly different from 0. So this is, currently running.
Meanwhile, I'll, dive. The solution of the of the last exercise so that we can still have a quick chat.
And maybe if there are additional follow-up questions. And So basically, here you see whenever it's colored, the TRF is significantly different from 0 and especially if you have lots of participants.
And this is quite prominent. You see the that for so channels sometimes when it's very close to 0 it's very close to 0 it's not significant.
Okay.
Maybe I should just draw 2 features.
Or 6.
Okay, yeah.
. One
So.
Subjects. Let me send
So.
So. Maybe also Here's, we see both the feature of court entropy and, Word surprising.
This is rather a subtle effect and here this is this for me because this is for me the first time that I try to see this data set.
Normally we expect this effect slightly earlier. But. Now I created. Created linguistic features using a completely different corpus.
So maybe that's, does the trick. Maybe this also It's likely, of course.
Okay.
Which will be a bit, of being, around. We did split again as expected, learning we seek features they have, this characteristic topography, Yeah.
As expected. I think for sake of time, I'll stop here. I'll add the, add some more codes and later to show this in one channel selection and to do as statistical testing and to see whenever the TRF is significant but I'll add that so you so you will have that as well.
Just to end this, workshop with, I'm sorry that it went super fast space.
I expected that it would have that it was virtual but anyway somehow Hi, I slightly prepared too much. If this overwhelmed you or so then and you have further questions really don't hesitate to contact me.
Here you have my email address. You can also reach out to me on Twitter. I have put all my code on on my Gitlap.
I think you should be able to find this so basically it's this repository so you can every generate everything from scratch.
Also, I try to document the code and my steps as as good as possible, but if you still have questions then don't hesitate to ask them now or to reach out to me later and It's a bit said that it's online it's also really cool that we have that you can all participants but I wish we could have a chance together.
I could see your faces and maybe grab a beer or so. So if we if you ever at the conference that I'm there too then please come and reach out and then we can have a
I'd have a beer.
Me too. Thank you very much.
So there is any questions.
Go ahead.
I have a few. It's sort of It sort of has to do with what you were saying about, channel selection.
In the in the data that you presented earlier today. Do you usually do some kind of, cluster like you look at a specific cluster of channels do you just look at what is significant above chance like how do you usually make that selection and how do you prevent sort of the multiple, like when you do the statistics, the multiple test,
Yeah. So.
I am. I think there are 2 proper ways to set channel selections.
And either it's a priori, like even before you start processing your code, by looking at what previous studies looked at.
So because like you should Set your channel selection. In an unbiased way, so to speak.
If you look at the prediction accuracy, see where it's significant, so on. Then you're actually already pricing your TRF analysis on where it will be significant.
If you want to, bypass this, I think you can also, but then it's super nested cross validation wise just to.
Really test does the prediction accuracy where significantly higher than 0 in one partition where you where the data is done seen and then just that on the TRF, but that's That's very, very nested.
So I just like to stick with a priori channel selections.
Yeah.
And that brings me to another. It brings me to another sort of topic that wasn't really discussed a lot in the workshop in general, but our lab started using the RCA method.
To sort of filter. The data before, into this sort of components. And then we look, we sort of run our TRFs on those components.
Do you think like what have you had any experience? With sort of those methodologies. Do you have any thoughts about that?
So I only use this using, principal component analysis first and then applying TRS or first estimating TRFs and then applying PCA.
Because then in a way you do your channel selection already. Which is nice. Right?
Because you only have one component. Depending on which component like how you go to the components. Which method you use I use PCA that I, set or notes for any projects that I present now, but previously that I said subject specifically and then.
Especially for acoustics, the first component will be super similar across all participants and then I think it's safe to use in case of PCA the polarity the sign is arbitrary so you just need to make sure that all the signs are flipped but except for that I think that's a very valid approach to do as well.
Thank you.
Hey Georgia, I'm gonna run to the other meeting just to open it up. I'll leave you in here as host.
Yes.
I just made you host and you can end it on time or just few minutes and then we'll meet in the other room.
Yeah, yeah, sure, sure.
Thank you very much, Marlis. I'll talk to you later. Have a good one.
Okay, thanks, bye.
Thanks.
Okay.
So now there is like a final meeting discussion general discussion, but if you have specific questions for Marlis, I think that's the best moment.
And, yeah. She can interact with her in person now. That's great.
Can't do it.
We'll see if you have questions on your data that you want to collect, that you want to know.
Bye. I.
Yes, Jada, go ahead. Yep.
I have a question.
That unless these are about to, Marley's tutorials, it might be best to ask in the main one.
Or in in the next session but if this is for Marley specifically by all means ask away.
Me too.
I think she's the right person because I'm really to have papers a lot now that I'm preparing my study so
Yeah.
Okay. Yes, it's red. So like whenever you spend all the effort in writing, it's so nice to The people actually, thanks.
Yeah, well, I'm preparing it. Actually, I'm about to start to collect data for a Saudi where we are focusing on product data for a study where we are focusing on prosody processing.
In people with the and also healthy controls. My question is As a continuous measure that I can beat into the to the TRF.
Is it a good idea to use, the continuous speech contour?
Yeah, sure. Like, I am probably you're aware, of, our group, but we also have, So data out of persons with a Asia listening to, continuous speech.
Yeah, yeah, yeah, yeah, I have that in mind
Yeah, I think, using like the acoustic features is a very good idea. Yeah.
Okay.
The only issue. Like it's the same for older adults as aphasia. It's always tricky to balance between yeah, your motivation and attention of the participants and the, and your amount of data that you can collect.
Ideally, you have as much data as possible, but for, persons with a Asia, it's, That's very trivial to do.
Like. Joe and Peter who collected like quite a lot of data from a patient, a, they had to exclude participants because they were literally falling asleep.
So So.
Yeah, yeah, hopefully our patient won't be. Serious, but people with very, very mild impairments.
So hopefully they will able to be attentive for a decent amount of time.
I really hope so.
Yeah, hopefully. All the best with your, with your study.
Thank you.
Good.
Okay, so go ahead, Georgia.
No, just mentioning that maybe we should move to the to the other room for the final discussion.
Yeah. We'll need that in this here. Yeah, because other people can't get into the other room because We are hosting this.
Right, right, true. Yep.
AlrightThank