[speaker003:] OK. So uh, he's not here,
[speaker004:] So.
[speaker003:] so you get to [disfmarker]
[speaker004:] Yeah, I will try to explain the thing that I did this [disfmarker] this week [disfmarker] during this week.
[speaker003:] Yeah.
[speaker004:] Well eh you know that I work [disfmarker] I begin to work with a new feature to detect voice unvoice.
[speaker005:] Mm hmm.
[speaker004:] What I trying two MLP to [disfmarker] to the [disfmarker] with this new feature and the fifteen feature uh from the eh bus base system
[speaker005:] The [disfmarker] the mel cepstrum?
[speaker004:] No, satly the mes the Mel Cepstrum, the new base system [disfmarker] the new base system.
[speaker005:] Oh the [disfmarker] OK,
[speaker004:] Yeah, we [disfmarker]
[speaker005:] the Aurora system.
[speaker004:] yeah the Aurora system with the new filter, VAD or something like that.
[speaker005:] OK.
[speaker004:] And I'm trying two MLP, one one that only have t three output, voice, unvoice, and silence,
[speaker003:] Mm hmm.
[speaker004:] and other one that have fifty six output. The probabilities of the allophone. And I tried to do some experiment of recognition with that and only have result with [disfmarker] with the MLP with the three output. And I put together the fifteen features and the three MLP output. And, well, the result are li a little bit better, but more or less similar.
[speaker003:] Uh, I [disfmarker] I'm [disfmarker] I'm slightly confused.
[speaker005:] Hmm.
[speaker003:] What [disfmarker] what feeds the uh [disfmarker] the three output net?
[speaker004:] Voice, unvoice, and si
[speaker003:] No no, what feeds it? What features does it see?
[speaker004:] The feature [disfmarker] the input? The inputs are the fifteen [disfmarker] the fifteen uh bases feature.
[speaker003:] Uh huh.
[speaker004:] the [disfmarker] with the new code. And the other three features are R, the variance of the difference between the two spectrum,
[speaker003:] Uh huh.
[speaker004:] the variance of the auto correlation function, except the [disfmarker] the first point, because half the height value is R zero
[speaker003:] Mm hmm. Mm hmm. Mm hmm. Mm hmm.
[speaker004:] and also R zero, the first coefficient of the auto correlation function. That is like the energy with these three feature,
[speaker003:] Right.
[speaker004:] also these three feature.
[speaker003:] You wouldn't do like R one over R zero or something like that? I mean usually for voiced unvoiced you'd do [disfmarker] yeah, you'd do something [disfmarker] you'd do energy
[speaker004:] Yeah.
[speaker003:] but then you have something like spectral slope, which is you get like R one ov over R zero or something like that.
[speaker004:] Uh yeah.
[speaker005:] What are the R's? I'm sorry I missed it.
[speaker003:] R correlations.
[speaker004:] No,
[speaker005:] Oh.
[speaker004:] R c No. Auto correlation? Yes, yes, the variance of the auto correlation function that uses that
[speaker003:] Ye Well that's the variance, but if you just say "what is [disfmarker]" I mean, to first order, um yeah one of the differences between voiced, unvoiced and silence is energy.
[speaker004:] Yeah,
[speaker003:] Another one is [disfmarker] but the other one is the spectral shape.
[speaker004:] I I'll [disfmarker] The spectral shape, yeah.
[speaker003:] Yeah, and so R one over R zero is what you typically use for that.
[speaker004:] No, I don't use that [disfmarker] I can't use [disfmarker]
[speaker003:] No, I'm saying that's what people us typically use.
[speaker004:] Mmm.
[speaker003:] See, because it [disfmarker] because this is [disfmarker] this is just like a single number to tell you um "does the spectrum look like that or does it look like that".
[speaker004:] Mm hmm.
[speaker001:] Oh. R [disfmarker] R [disfmarker] R zero.
[speaker003:] Right?
[speaker004:] Mm hmm.
[speaker003:] So if it's [disfmarker] if it's um [disfmarker] if it's low energy uh but the [disfmarker] but the spectrum looks like that or like that, it's probably silence.
[speaker004:] Mm hmm.
[speaker003:] Uh but if it's low energy and the spectrum looks like that, it's probably unvoiced.
[speaker004:] Yeah.
[speaker003:] So if you just [disfmarker] if you just had to pick two features to determine voiced unvoiced, you'd pick something about the spectrum like uh R one over R zero, um and R zero
[speaker004:] Mm hmm, OK.
[speaker003:] or i i you know you'd have some other energy measure and like in the old days people did like uh zero crossing counts.
[speaker004:] Yeah, yeah.
[speaker003:] Right. S S
[speaker004:] Well, I can also th use this.
[speaker003:] Yeah. Um,
[speaker004:] Bec because the result are a little bit better but we have in a point that everything is more or less the similar [disfmarker] more or less similar.
[speaker003:] Yeah. But um
[speaker004:] It's not quite better.
[speaker003:] Right, but it seemed to me that what you were what you were getting at before was that there is something about the difference between the original signal or the original FFT and with the filter which is what [disfmarker] and the variance was one take uh on it.
[speaker004:] Yeah, I used this too.
[speaker003:] Right. But it [disfmarker] it could be something else. Suppose you didn't have anything like that. Then in that case, if you have two nets, Alright, and this one has three outputs, and this one has f
[speaker004:] Mm hmm.
[speaker003:] whatever, fifty six, or something,
[speaker004:] Mm hmm.
[speaker003:] if you were to sum up the probabilities for the voiced and for the unvoiced and for the silence here, we've found in the past you'll do better at voiced unvoiced silence than you do with this one. So just having the three output thing doesn't [disfmarker] doesn't really buy you anything.
[speaker004:] Yeah.
[speaker003:] The issue is what you feed it.
[speaker004:] Yeah, I have [disfmarker] yeah.
[speaker003:] So uh
[speaker005:] So you're saying take the features that go into the voiced unvoiced silence net and feed those into the other one, as additional inputs, rather than having a separate [disfmarker]
[speaker004:] No [disfmarker]
[speaker003:] w W well that's another way.
[speaker004:] Yeah.
[speaker003:] That wasn't what I was saying but yeah that's certainly another thing to do. No I was just trying to say if you b if you bring this into the picture over this, what more does it buy you?
[speaker005:] Mmm.
[speaker003:] And what I was saying is that the only thing I think that it buys you is um based on whether you feed it something different. And something different in some fundamental way. And so the kind of thing that [disfmarker] that she was talking about before, was looking at something uh ab um [disfmarker] something uh about the difference between the [disfmarker] the uh um log FFT uh log power uh and the log magnitude uh F F spectrum uh and the um uh filter bank.
[speaker004:] Yeah.
[speaker003:] And so the filter bank is chosen in fact to sort of integrate out the effects of pitch and she's saying you know trying [disfmarker] So the particular measure that she chose was the variance of this m of this difference,
[speaker004:] Mm hmm.
[speaker003:] but that might not be the right number.
[speaker004:] Maybe.
[speaker003:] Right? I mean maybe there's something about the variance that's [disfmarker] that's not enough or maybe there's something else that [disfmarker] that one could use, but I think that, for me, the thing that [disfmarker] that struck me was that uh you wanna get something back here, so here's [disfmarker] here's an idea. uh What about it you skip all the [disfmarker] all the really clever things, and just fed the log magnitude spectrum into this?
[speaker004:] Ah [disfmarker] I'm sorry.
[speaker003:] This is f You have the log magnitude spectrum, and you were looking at that and the difference between the filter bank and [disfmarker] and c c computing the variance.
[speaker004:] Yeah. Mm hmm. Mm hmm.
[speaker003:] That's a clever thing to do. What if you stopped being clever?
[speaker004:] Mm hmm.
[speaker003:] And you just took this thing in here because it's a neural net and neural nets are wonderful and figure out what they can [disfmarker] what they most need from things, and I mean that's what they're good at.
[speaker004:] Yeah.
[speaker003:] So I mean you're [disfmarker] you're [disfmarker] you're trying to be clever and say what's the statistic that should [disfmarker] we should get about this difference but uh in fact, you know maybe just feeding this in or [disfmarker] or feeding both of them in
[speaker005:] Hmm.
[speaker003:] you know, another way, saying let it figure out what's the [disfmarker] what is the interaction, especially if you do this over multiple frames?
[speaker004:] Mm hmm.
[speaker003:] Then you have this over time, and [disfmarker] and both kinds of measures and uh you might get uh something better.
[speaker004:] Mm hmm.
[speaker003:] Um.
[speaker005:] So [disfmarker] so don't uh [disfmarker] don't do the division, but let the net have everything.
[speaker003:] That's another thing you could do yeah. Yeah.
[speaker004:] Yeah.
[speaker003:] Um. I mean, it seems to me, if you have exactly the right thing then it's better to do it without the net because otherwise you're asking the net to learn this [disfmarker] you know, say if you wanted to learn how to do multiplication.
[speaker005:] Mm hmm.
[speaker003:] I mean you could feed it a bunch of s you could feed two numbers that you wanted to multiply into a net and have a bunch of nonlinearities in the middle and train it to get the product of the output and it would work. But, it's kind of crazy, cuz we know how to multiply and you [disfmarker] you'd be you know much lower error usually [vocalsound] if you just multiplied it out. But suppose you don't really know what the right thing is. And that's what these sort of dumb machine learning methods are good at. So. Um. Anyway. It's just a thought.
[speaker005:] How long does it take, Carmen, to train up one of these nets?
[speaker004:] Oh, not too much.
[speaker005:] Yeah.
[speaker004:] Mmm, one day or less.
[speaker005:] Hmm.
[speaker003:] Yeah, it's probably worth it.
[speaker001:] What are [disfmarker] what are your f uh frame error rates for [disfmarker] for this?
[speaker004:] Eh fifty f six uh no, the frame error rate? Fifty six I think.
[speaker001:] O
[speaker003:] Is that [disfmarker] maybe that's accuracy?
[speaker004:] Percent. The accuracy.
[speaker001:] Fif fifty six percent accurate for v voice unvoice
[speaker004:] Mm hmm. No for, yes f I don't remember for voice unvoice,
[speaker001:] Oh, OK.
[speaker004:] maybe for the other one.
[speaker003:] Yeah, voiced unvoiced hopefully would be a lot better.
[speaker001:] OK.
[speaker004:] for voiced. I don't reme
[speaker001:] Should be in nineties somewhere.
[speaker004:] Better. Maybe for voice unvoice.
[speaker001:] Right.
[speaker004:] This is for the other one. I should [disfmarker] I can't show that.
[speaker001:] OK.
[speaker004:] But I think that fifty five was for the [disfmarker] when the output are the fifty six phone.
[speaker001:] Mm hmm.
[speaker004:] That I look in the [disfmarker] with the other [disfmarker] nnn the other MLP that we have are more or less the same number. Silence will be better but more or less the same.
[speaker003:] I think at the frame level for fifty six that was the kind of number we were getting for [disfmarker] for uh um reduced band width uh stuff.
[speaker004:] I think that [disfmarker] I [disfmarker] I [disfmarker] I think that for the other one, for the three output, is sixty sixty two, sixty three more or less.
[speaker001:] Mm hmm.
[speaker003:] That's all?
[speaker004:] It's [disfmarker] Yeah.
[speaker003:] That's pretty bad.
[speaker004:] Yeah, because it's noise also.
[speaker003:] Aha!
[speaker001:] Oh yeah.
[speaker004:] And we have
[speaker003:] Aha! Yeah.
[speaker004:] I know.
[speaker003:] Yeah. OK. But even i in [disfmarker] Oh yeah, in training. Still, Uh. Well actually, so this is a test that you should do then. Um, if you're getting fifty six percent over here, uh that's in noise also, right?
[speaker004:] Yeah, yeah, yeah.
[speaker003:] Oh OK. If you're getting fifty six here, try adding together the probabilities of all of the voiced phones here and all of the unvoiced phones
[speaker004:] will be [disfmarker]
[speaker003:] and see what you get then.
[speaker004:] Yeah.
[speaker003:] I bet you get better than sixty three.
[speaker004:] Well I don't know, but [disfmarker] I th I [disfmarker] I think that we [disfmarker] I have the result more or less. Maybe. I don't know. I don't [disfmarker] I'm not sure but I remember [@ @] that I can't show that.
[speaker003:] OK, but that's a [disfmarker] That is a [disfmarker] a good check point, you should do that anyway,
[speaker004:] Yeah.
[speaker003:] OK? Given this [disfmarker] this uh regular old net that's just for choosing for other purposes, uh add up the probabilities of the different subclasses and see [disfmarker] see how well you do. Uh and that [disfmarker] you know anything that you do over here should be at least as good as that.
[speaker004:] Mm hmm.
[speaker003:] OK.
[speaker004:] I will do that.
[speaker005:] The targets for the neural net, uh, they come from forced alignments?
[speaker004:] But [disfmarker] Uh, [comment] no.
[speaker001:] TIMIT canonical ma mappings.
[speaker004:] TIMIT.
[speaker005:] Ah!
[speaker003:] Oh. So, this is trained on TIMIT.
[speaker005:] OK.
[speaker004:] Yeah.
[speaker001:] Yeah, noisy TIMIT.
[speaker003:] OK.
[speaker004:] Yeah this for TIMIT.
[speaker003:] But noisy TIMIT?
[speaker004:] Noisy TIMIT.
[speaker001:] Right.
[speaker004:] We have noisy TIMIT with the noise of the [disfmarker] the TI digits. And now we have another noisy TIMIT also with the noise of uh Italian database.
[speaker003:] I see. Yeah. Well there's gonna be [disfmarker] it looks like there's gonna be a noisy uh [disfmarker] some large vocabulary noisy stuff too. Somebody's preparing.
[speaker005:] Really?
[speaker003:] Yeah. I forget what it'll be, resource management, Wall Street Journal, something. Some [disfmarker] some read task actually, that they're [disfmarker] preparing.
[speaker001:] Hmm!
[speaker005:] For what [disfmarker] For Aurora?
[speaker003:] Yeah.
[speaker005:] Oh!
[speaker003:] Yeah, so the uh [disfmarker] Uh, the issue is whether people make a decision now based on what they've already seen, or they make it later. And one of the arguments for making it later is let's make sure that whatever techniques that we're using work for something more than [disfmarker] than connected digits.
[speaker005:] Hmm.
[speaker003:] So.
[speaker005:] When are they planning [disfmarker] When would they do that?
[speaker003:] Mmm, I think late [disfmarker] uh I think in the summer sometime.
[speaker005:] Hmm.
[speaker003:] So. OK, thanks.
[speaker004:] This is the work that I did during this date
[speaker003:] Uh huh.
[speaker004:] and also mmm I [disfmarker] H Hynek last week say that if I have time I can to begin to [disfmarker] to study well seriously the France Telecom proposal
[speaker003:] Mm hmm.
[speaker004:] to look at the code and something like that to know exactly what they are doing because maybe that we can have some ideas
[speaker003:] Mm hmm.
[speaker004:] but not only to read the proposal. Look insi look i carefully what they are doing with the program [@ @] and I begin to [disfmarker] to work also in that. But the first thing that I don't understand is that they are using R the uh log energy that this quite [disfmarker] I don't know why they have some constant in the expression of the lower energy. I don't know what that means.
[speaker005:] They have a constant in there, you said?
[speaker004:] Yeah.
[speaker003:] Oh, at the front it says uh "log energy is equal to the rounded version of sixteen over the log of two"
[speaker004:] This [disfmarker] Yeah.
[speaker003:] Uh.
[speaker004:] Then maybe I can understand.
[speaker003:] uh times the [disfmarker] Well, this is natural log, and maybe it has something to do with the fact that this is [disfmarker]
[speaker005:] Is that some kind of base conversion,
[speaker003:] I [disfmarker] I have no idea.
[speaker005:] or [disfmarker]?
[speaker003:] Yeah, that's what I was thinking, but [disfmarker] but um, then there's the sixty four, Uh, [vocalsound] I don't know.
[speaker004:] Because maybe they're [disfmarker] the threshold that they are using on the basis of this value [disfmarker]
[speaker005:] Experimental results.
[speaker001:] Mc McDonald's constant.
[speaker004:] I don't know exactly, because well th I thought maybe they have a meaning. But I don't know what is the meaning of take exactly this value.
[speaker003:] Yeah, it's pretty funny looking.
[speaker005:] So they're taking the number inside the log and raising it to sixteen over log base two.
[speaker003:] I don't know. Yeah, I [disfmarker] um Right. Sixteen over [comment] two.
[speaker005:] Does it have to do with those sixty fours, or [disfmarker]?
[speaker003:] Um. If we ignore the sixteen, the natural log of t one over the natural log of two times the natu I don't know. Well, maybe somebody'll think of something, but this is uh [disfmarker] It may just be that they [disfmarker] they want to have [disfmarker] for very small energies, they want to have some kind of a [disfmarker]
[speaker004:] Yeah, the e The effect I don't [disfmarker] [@ @] I can understand the effect of this, no? because it's to [disfmarker] to do something like that. No?
[speaker003:] Well, it says, since you're taking a natural log, it says that when [disfmarker] when you get down to essentially zero energy, this is gonna be the natural log of one, which is zero.
[speaker004:] Mm hmm.
[speaker003:] So it'll go down to uh to [nonvocalsound] the natural log being [disfmarker] So the lowest value for this would be zero. So y you're restricted to being positive. And this sort of smooths it for very small energies. Uh, why they chose sixty four and something else, that was probably just experimental.
[speaker004:] Yeah.
[speaker003:] And the [disfmarker] the [disfmarker] the constant in front of it, I have no idea. um
[speaker004:] Well. I [disfmarker] I will look to try if I move this parameter in their code what happens, maybe everything is [disfmarker] Maybe they tres hole are on basis of this.
[speaker003:] uh [disfmarker] I mean [disfmarker] it [disfmarker] [vocalsound] they [disfmarker] they probably have some fi particular s fixed point arithmetic that they're using,
[speaker004:] I don't know.
[speaker005:] Yeah, I was just gonna say maybe it has something to do with hardware,
[speaker003:] and then it just [disfmarker]
[speaker005:] something they were doing.
[speaker003:] Yeah. Yeah, I mean that [disfmarker] they're s probably working with fixed point or integer or something. I think you're supposed to on this stuff anyway, and [disfmarker] and so maybe that puts it in the right realm somewhere.
[speaker005:] Well it just, yeah, puts it in the right range, or [disfmarker]
[speaker003:] Yeah. I think, given at the level you're doing things in floating point on the computer, I don't think it matters, would be my guess,
[speaker004:] Mm hmm.
[speaker003:] but.
[speaker004:] I [disfmarker] this more or less anything
[speaker003:] Yeah. OK, and wh when did Stephane take off? He took off [disfmarker]
[speaker004:] I think that Stephane will arrive today or tomorrow.
[speaker003:] Oh, he was gone these first few days, and then he's here for a couple days before he goes to Salt Lake City.
[speaker004:] Mm hmm.
[speaker003:] OK.
[speaker004:] He's [disfmarker] I think that he is in Las Vegas or something like that.
[speaker003:] Yeah. Yeah. So he's [disfmarker] he's going to ICASSP which is good. I [disfmarker] I don't know if there are many people who are going to ICASSP
[speaker004:] Yeah.
[speaker003:] so [disfmarker] so I thought, make sure somebody go.
[speaker004:] Yeah.
[speaker005:] Do [disfmarker] have [disfmarker] Have people sort of stopped going to ICASSP in recent years?
[speaker003:] Um, people are less consistent about going to ICASSP and I think it's still [disfmarker] it's still a reasonable forum for students to [disfmarker] to present things. Uh, it's [disfmarker] I think for engineering students of any kind, I think it's [disfmarker] it's if you haven't been there much, it's good to go to, uh to get a feel for things, a range of things, not just speech. Uh. But I think for [disfmarker] for sort of dyed in the wool speech people, um I think that ICSLP and Eurospeech are much more targeted.
[speaker005:] Mm hmm.
[speaker003:] Uh. And then there's these other meetings, like HLT and [disfmarker] and uh ASRU [disfmarker] so there's [disfmarker] there's actually plenty of meetings that are really relevant to [disfmarker] to uh computational uh speech processing of one sort or another.
[speaker005:] Mm hmm.
[speaker003:] Um. So. I mean, I mostly just ignored it because I was too busy and [vocalsound] didn't get to it. So uh Wanna talk a little bit about what we were talking about this morning?
[speaker001:] Oh!
[speaker003:] Just briefly, or [pause] Or anything else?
[speaker001:] um [pause] uh [pause] Yeah. So. I [disfmarker] I guess some of the progress, I [disfmarker] I've been getting a [disfmarker] getting my committee members for the quals. And um so far I have Morgan and Hynek, [vocalsound] Mike Jordan, and I asked John Ohala and he agreed.
[speaker005:] Cool.
[speaker001:] Yeah. Yeah. So I'm [disfmarker] I [disfmarker] I just need to ask um Malek. One more. Um. Tsk. Then uh I talked a little bit about [vocalsound] um continuing with these dynamic ev um acoustic events, and um [vocalsound] [vocalsound] we're [disfmarker] we're [disfmarker] we're [vocalsound] thinking about a way to test the completeness of a [disfmarker] a set of um dynamic uh events. Uh, completeness in the [disfmarker] in the sense that [vocalsound] um if we [disfmarker] if we pick these X number of acoustic events, [vocalsound] do they provide sufficient coverage [vocalsound] for the phones that we're trying to recognize [vocalsound] or [disfmarker] or the f the words that we're gonna try to recognize later on. And so Morgan and I were uh discussing [vocalsound] um s uh s a form of a cheating experiment [vocalsound] where we get [disfmarker] [vocalsound] um we have uh [vocalsound] um a chosen set of features, or acoustic events, and we train up a hybrid [vocalsound] um system to do phone recognition on TIMIT. So i i the idea is if we get good phone recognition results, [vocalsound] using um these set of acoustic events, [vocalsound] then [vocalsound] um that [disfmarker] that says that these acoustic events are g sufficient to cover [vocalsound] a set of phones, at least found in TIMIT. Um so i it would be a [disfmarker] [vocalsound] a measure of "are we on the right track with [disfmarker] with the [disfmarker] the choices of our acoustic events". Um, [vocalsound] So that's going on. And [vocalsound] also, just uh working on my [vocalsound] uh final project for Jordan's class, uh which is [disfmarker]
[speaker003:] Actually, let me [disfmarker] Hold that thought.
[speaker001:] Yeah.
[speaker003:] Let me back up while we're still on it.
[speaker001:] OK, sure.
[speaker003:] The [disfmarker] the other thing I was suggesting, though, is that given that you're talking about binary features, uh, maybe the first thing to do is just to count and uh count co occurrences and get probabilities for a discrete HMM cuz that'd be pretty simple because it's just [disfmarker] Say, if you had ten [disfmarker] ten events, uh that you were counting, uh each frame would only have a thousand possible values for these ten bits, and uh so you could make a table that would [disfmarker] say, if you had thirty nine phone categories, that would be a thousand by thirty nine, and just count the co occurrences and divide them by the [disfmarker] the uh [disfmarker] uh uh occ uh count the co occurrences between the event and the phone and divide them by the number of occurrences of the phone, and that would give you the likelihood of the [disfmarker] of the event given the phone. And um then just use that in a very simple HMM and uh you could uh do phone recognition then and uh wouldn't have any of the issues of the uh training of the net or [disfmarker] I mean, it'd be on the simple side, but
[speaker005:] Mm hmm.
[speaker003:] uh um you know, if [disfmarker] uh uh the example I was giving was that if [disfmarker] if you had um onset of voicing and [disfmarker] and end of voicing as being two kinds of events, then if you had those a all marked correctly, and you counted co occurrences, you should get it completely right.
[speaker005:] Mm hmm.
[speaker003:] So. um [disfmarker] But you'd get all the other distinctions, you know, randomly wrong. I mean there'd be nothing to tell you that. So um [vocalsound] uh If you just do this by counting, then you should be able to find out in a pretty straightforward way whether you have a sufficient uh set of events to [disfmarker] to do the kind of level of [disfmarker] [vocalsound] of uh classification of phones that you'd like. So that was [disfmarker] that was the idea. And then the other thing that we were discussing was [disfmarker] was um [vocalsound] OK, how do you get the [disfmarker] your training data.
[speaker005:] Mm hmm.
[speaker003:] Cuz uh the [vocalsound] Switchboard transcription project uh uh you know was half a dozen people, or so working off and on over a couple years, and uh similar [disfmarker] [vocalsound] similar amount of data [vocalsound] to what you're talking about with TIMIT training. So, it seems to me that the only reasonable starting point is uh to automatically translate the uh current TIMIT markings into the markings you want. And uh [vocalsound] it won't have the kind of characteristic that you'd like, of catching funny kind of things that maybe aren't there from these automatic markings,
[speaker005:] Mm hmm.
[speaker003:] but [disfmarker] but uh it's uh [disfmarker]
[speaker005:] It's probably a good place to start.
[speaker003:] Yeah.
[speaker005:] Yeah.
[speaker003:] Yeah and a short [disfmarker] short amount of time, just to [disfmarker] again, just to see if that information is sufficient to uh determine the phones.
[speaker005:] Mm hmm. Hmm.
[speaker003:] So.
[speaker005:] Yeah, you could even then [disfmarker] to [disfmarker] to get an idea about how different it is, you could maybe take some subset and you know, go through a few sentences, mark them by hand and then see how different it is from you know, the canonical ones,
[speaker003:] Right.
[speaker005:] just to get an idea [disfmarker] a rough idea of h if it really even makes a difference.
[speaker003:] You can get a little feeling for it that way,
[speaker005:] Yeah.
[speaker003:] yeah that is probably right. I mean uh my [disfmarker] my guess would be that this is [disfmarker] since TIMIT's read speech that this would be less of a big deal,
[speaker005:] Mm hmm.
[speaker003:] if you went and looked at spontaneous speech it'd be more [disfmarker] more of one.
[speaker005:] Right. Right.
[speaker003:] And the other thing would be, say, if you had these ten events, you'd wanna see, well what if you took two events or four events or ten events or t and you know, and [disfmarker] and hopefully there should be some point at which [vocalsound] having more information doesn't tell you really all that much more about what the phones are.
[speaker005:] Mm hmm. You could define other events as being sequences of these events too.
[speaker003:] Uh, you could, but the thing is, what he's talking about here is a uh [disfmarker] a translation to a per frame feature vector, so there's no sequence in that, I think. I think it's just a [disfmarker]
[speaker005:] Unless you did like a second pass over it or something after you've got your [disfmarker]
[speaker003:] Yeah, but we're just talking about something simple here, yeah, to see if [disfmarker]
[speaker005:] Yeah. Yeah, yeah. Yeah. I'm adding complexity.
[speaker003:] Yeah. Just [disfmarker] You know. The idea is with a [disfmarker] with a very simple statistical structure, could you [disfmarker] could you uh at least verify that you've chosen features that [vocalsound] are sufficient.
[speaker005:] Yeah.
[speaker003:] OK, and you were saying something [disfmarker] starting to say something else about your [disfmarker] your class project, or [disfmarker]?
[speaker001:] Oh. Yeah th Um.
[speaker003:] Yeah.
[speaker001:] So for my class project I'm [vocalsound] um [vocalsound] [vocalsound] I'm tinkering with uh support vector machines? something that we learned in class, and uh um basically just another method for doing classification. And so I'm gonna apply that to [vocalsound] um compare it with the results by um King and Taylor who did [vocalsound] um these um using recurrent neural nets, they recognized [vocalsound] um [vocalsound] a set of phonological features um and made a mapping from the MFCC's to these phonological features, so I'm gonna [vocalsound] do a similar thing with [disfmarker] [vocalsound] with support vector machines and see if [disfmarker]
[speaker005:] So what's the advantage of support vector machines? What [disfmarker]
[speaker001:] Um. So, support vector machines are [disfmarker] are good with dealing with a less amount of data
[speaker005:] Hmm.
[speaker001:] and um so if you [disfmarker] if you give it less data it still does a reasonable job [vocalsound] in learning the [disfmarker] the patterns.
[speaker005:] Hmm.
[speaker001:] Um and [vocalsound] um
[speaker003:] I guess it [disfmarker] yeah, they're sort of succinct, and [disfmarker] and they [vocalsound] uh
[speaker001:] Yeah.
[speaker005:] Does there some kind of a distance metric that they use or how do they [disfmarker] for cla what do they do for classification?
[speaker001:] Um. Right. So, [vocalsound] the [disfmarker] the simple idea behind a support vector machine is [vocalsound] um, [vocalsound] you have [disfmarker] you have this feature space, right?
[speaker005:] Mm hmm.
[speaker001:] and then it finds the optimal separating plane, um between these two different um classes,
[speaker005:] Mm hmm. Mm hmm.
[speaker001:] and um [vocalsound] and so [vocalsound] um, what it [disfmarker] i at the end of the day, what it actually does is [vocalsound] it picks [vocalsound] those examples of the features that are closest to the separating boundary, and remembers those
[speaker005:] Mm hmm.
[speaker001:] and [disfmarker] [vocalsound] and uses them to recreate the boundary for the test set. So, given these [vocalsound] um these features, or [disfmarker] or these [disfmarker] these examples, [pause] um, [pause] critical examples, [vocalsound] which they call support f support vectors, [vocalsound] then um [vocalsound] given a new example, [vocalsound] if the new example falls [vocalsound] um away from the boundary in one direction then it's classified as being a part of this particular class
[speaker005:] Oh.
[speaker001:] and otherwise it's the other class.
[speaker005:] So why save the examples? Why not just save what the boundary itself is?
[speaker001:] Mm hmm. Um. Hmm. Let's see. Uh. Yeah, that's a good question. I [disfmarker]
[speaker003:] That's another way of doing it.
[speaker001:] yeah.
[speaker005:] Mmm.
[speaker003:] Right? So [disfmarker] so it [disfmarker] I mean I [disfmarker] I guess it's [disfmarker]
[speaker005:] Sort of an equivalent.
[speaker003:] You know, it [disfmarker] it goes back to nearest neighbor [vocalsound] sort of thing,
[speaker005:] Mm hmm.
[speaker003:] right? Um, i i if [disfmarker] is it eh w When is nearest neighbor good? Well, nearest neighbor good [disfmarker] is good if you have lots and lots of examples. Um but of course if you have lots and lots of examples, then it can take a while to [disfmarker] to use nearest neighbor. There's lots of look ups. So a long time ago people talked about things where you would have uh a condensed nearest neighbor, where you would [disfmarker] you would [disfmarker] you would pick out uh some representative examples which would uh be sufficient to represent [disfmarker] to [disfmarker] to correctly classify everything that came in.
[speaker005:] Oh. Mm hmm.
[speaker003:] I [disfmarker] I think s I think support vector stuff sort of goes back to [disfmarker] [vocalsound] to that kind of thing.
[speaker005:] I see.
[speaker003:] Um.
[speaker005:] So rather than doing nearest neighbor where you compare to every single one, you just pick a few critical ones, and [disfmarker]
[speaker003:] Yeah.
[speaker005:] Hmm.
[speaker003:] And th the You know, um neural net approach uh or Gaussian mixtures for that matter are sort of [disfmarker] fairly brute force kinds of things, where you sort of [disfmarker] [vocalsound] you predefine that there is this big bunch of parameters and then you [disfmarker] you place them as you best can to define the boundaries, and in fact, as you know, [vocalsound] these things do take a lot of parameters and [disfmarker] and uh [vocalsound] if you have uh only a modest amount of data, you have trouble [vocalsound] uh learning them. Um, so I [disfmarker] I guess the idea to this is that it [disfmarker] it is reputed to uh be somewhat better in that regard.
[speaker005:] Mm hmm.
[speaker001:] Right. I it can be a [disfmarker] a reduced um [vocalsound] parameterization of [disfmarker] of the [disfmarker] the model by just keeping [vocalsound] certain selected examples.
[speaker005:] Hmm.
[speaker001:] Yeah. So.
[speaker003:] But I don't know if people have done sort of careful comparisons of this on large tasks or anything. Maybe [disfmarker] maybe they have. I don't know.
[speaker001:] Yeah, I don't know either.
[speaker003:] Yeah.
[speaker002:] S do you get some kind of number between zero and one at the output?
[speaker001:] Actually you don't get a [disfmarker] you don't get a nice number between zero and one. You get [disfmarker] you get either a zero or a one. Um, uh there are [disfmarker] there are pap Well, basically, it's [disfmarker] it's um [vocalsound] you [disfmarker] you get a distance measure at the end of the day, and then that distance measure is [disfmarker] is um [disfmarker] [vocalsound] is translated to a zero or one. Um.
[speaker003:] But that's looking at it for [disfmarker] for classification [disfmarker] for binary classification,
[speaker001:] That's for classification,
[speaker005:] And you get that for each class, you get a zero or a one.
[speaker003:] right?
[speaker001:] right. Right.
[speaker003:] But you have the distances to work with.
[speaker001:] You have the distances to work with,
[speaker003:] Cuz actually Mississippi State people did use support vector machines for uh uh speech recognition and they were using it to estimate probabilities.
[speaker001:] yeah. Yeah. Yeah, they [disfmarker] [vocalsound] they had a [disfmarker] had a way to translate the distances into [disfmarker] into probabilities with the [disfmarker] with the simple [vocalsound] um [vocalsound] uh sigmoidal function.
[speaker003:] Yeah, and d did they use sigmoid or a softmax type thing? And didn't they like exponentiate or something
[speaker001:] Um [pause] [vocalsound] Yeah,
[speaker003:] and then [vocalsound] divide by the sum of them,
[speaker001:] there's some [disfmarker] there's like one over one plus the exponential or something like that.
[speaker003:] or [disfmarker]? Oh it [disfmarker] i Oh, so it is a sigmoidal.
[speaker001:] Yeah.
[speaker003:] OK. Alright.
[speaker005:] Did the [disfmarker] did they get good results with that?
[speaker003:] I mean, they're OK, I [disfmarker] I don't [disfmarker] I don't think they were earth [disfmarker] earth shattering, but I think that [vocalsound] uh this was a couple years ago,
[speaker005:] Hmm.
[speaker003:] I remember them doing it at some meeting, and [disfmarker] and um I don't think people were very critical because it was interesting just to [disfmarker] to try this and you know, it was the first time they tried it, so [disfmarker] [vocalsound] so the [disfmarker] you know, the numbers were not incredibly good
[speaker005:] Hmm.
[speaker003:] but there's you know, it was th reasonable.
[speaker005:] Mm hmm.
[speaker003:] I [disfmarker] I don't remember anymore. I don't even remember what the task was, it [comment] was Broadcast News, or [vocalsound] something. I don't know.
[speaker005:] Hmm.
[speaker001:] Right.
[speaker002:] Uh s So Barry, if you just have zero and ones, how are you doing the speech recognition?
[speaker001:] Oh I'm not do I'm not planning on doing speech recognition with it. I'm just doing [vocalsound] detection of phonological features.
[speaker002:] Oh. OK.
[speaker001:] So uh for example, [vocalsound] this [disfmarker] this uh feature set called the uh sound patterns of English [vocalsound] um is just a bunch of [vocalsound] um [vocalsound] binary valued features. Let's say, is this voicing, or is this not voicing, is this [vocalsound] sonorants, not sonorants, and [vocalsound] stuff like that.
[speaker002:] OK.
[speaker001:] So.
[speaker005:] Did you find any more mistakes in their tables?
[speaker001:] Oh! Uh I haven't gone through the entire table, [pause] yet. Yeah, yesterday I brought Chuck [vocalsound] the table and I was like, "wait, this [disfmarker] is [disfmarker] Is the mapping from N to [disfmarker] to this phonological feature called um" coronal ", is [disfmarker] is [disfmarker] should it be [disfmarker] shouldn't it be a one? or should it [disfmarker] should it be you know coronal instead of not coronal as it was labelled in the paper? So I ha haven't hunted down all the [disfmarker] all the mistakes yet,
[speaker003:] Uh huh.
[speaker001:] but [disfmarker]
[speaker003:] But a as I was saying, people do get probabilities from these things,
[speaker002:] OK.
[speaker003:] and [disfmarker] and uh we were just trying to remember how they do, but people have used it for speech recognition, and they have gotten probabilities. So they have some conversion from these distances to probabilities.
[speaker002:] OK.
[speaker001:] Right, yeah.
[speaker003:] There's [disfmarker] you have [disfmarker] you have the paper, right? The Mississippi State paper?
[speaker001:] Mm hmm. Mm hmm.
[speaker003:] Yeah, if you're interested y you could look,
[speaker002:] And [disfmarker] OK. OK.
[speaker001:] Yeah,
[speaker003:] yeah.
[speaker001:] I can [disfmarker] I can show you [disfmarker] I [disfmarker] yeah,
[speaker005:] So in your [disfmarker] in [disfmarker] in the thing that you're doing, uh you have a vector of ones and zeros for each phone?
[speaker001:] our [disfmarker] Mm hmm. Uh, is this the class project, or [disfmarker]?
[speaker005:] Yeah.
[speaker001:] OK. um
[speaker005:] Is that what you're [disfmarker]
[speaker001:] Right, [comment] Right, right f so for every phone there is [disfmarker] there is a um [disfmarker] a vector of ones and zeros [vocalsound] f uh corresponding to whether it exhibits a particular phonological feature or not.
[speaker005:] Mm hmm. Mm hmm. And so when you do your wh I'm [disfmarker] what is the task for the class project? To come up with the phones?
[speaker001:] Um
[speaker005:] or to come up with these vectors to see how closely they match the phones,
[speaker001:] Oh. Right, um to come up with a mapping from um MFCC's or s some feature set, [vocalsound] um to [vocalsound] uh w to whether there's existence of a particular phonological feature.
[speaker005:] or [disfmarker]? Mm hmm.
[speaker001:] And um yeah, basically it's to learn a mapping [vocalsound] from [disfmarker] [vocalsound] from the MFCC's to [vocalsound] uh phonological features. Is it [disfmarker] did that answer your question?
[speaker005:] I think so.
[speaker001:] OK.
[speaker005:] I guess [disfmarker] I mean, uh [disfmarker] I'm not sure what you [disfmarker] what you're [disfmarker] what you get out of your system.
[speaker001:] C
[speaker005:] Do you get out a uh [disfmarker] a vector of these ones and zeros and then try to find the closest matching phoneme to that vector,
[speaker001:] Mm hmm.
[speaker005:] or [disfmarker]?
[speaker001:] Oh. No, no. I'm not [disfmarker] I'm not planning to do any [disfmarker] any phoneme mapping yet. Just [disfmarker] [vocalsound] it's [disfmarker] it's basically [disfmarker] it's [disfmarker] it's really simple, basically a detection [vocalsound] of phonological features.
[speaker005:] Uh huh. I see.
[speaker001:] Yeah, and um [vocalsound] [vocalsound] cuz the uh [disfmarker] So King and [disfmarker] and Taylor [vocalsound] um did this with uh recurrent neural nets,
[speaker005:] Yeah. Mm hmm.
[speaker001:] and this i their [disfmarker] their idea was to first find [vocalsound] a mapping from MFCC's to [vocalsound] uh phonological features and then later on, once you have these [vocalsound] phonological features, [vocalsound] then uh map that to phones.
[speaker005:] Mm hmm.
[speaker001:] So I'm [disfmarker] I'm sort of reproducing phase one of their stuff.
[speaker005:] Mmm. So they had one recurrent net for each particular feature?
[speaker001:] Right. Right. Right. Right.
[speaker005:] I see. I wo did they compare that [disfmarker] I mean, what if you just did phone recognition and did the reverse lookup.
[speaker001:] Uh.
[speaker005:] So you recognize a phone and which ever phone was recognized, you spit out it's vector of ones and zeros.
[speaker001:] Mm hmm.
[speaker003:] I expect you could do that.
[speaker005:] I mean uh [disfmarker]
[speaker001:] Uh.
[speaker003:] That's probably not what he's going to do on his class project.
[speaker005:] Yeah.
[speaker003:] Yeah.
[speaker005:] No.
[speaker003:] So um have you had a chance to do this um thing we talked about yet with the uh [disfmarker]
[speaker001:] Yeah.
[speaker003:] um
[speaker005:] Insertion penalty?
[speaker003:] Uh. No actually I was going a different [disfmarker] That's a good question, too, but I was gonna ask about the [disfmarker] [vocalsound] the um [vocalsound] changes to the data in comparing PLP and mel cepstrum for the SRI system.
[speaker005:] Uh. Well what I've been [disfmarker] "Changes to the data", I'm not sure I [disfmarker]
[speaker003:] Right. So we talked on the phone about this, that [disfmarker] that there was still a difference of a [disfmarker] of a few percent
[speaker005:] Yeah. Right.
[speaker003:] and [vocalsound] you told me that there was a difference in how the normalization was done. And I was asking if you were going to do [disfmarker] [vocalsound] redo it uh for PLP with the normalization done as it had been done for the mel cepstrum.
[speaker005:] Mm hmm. Uh right, no I haven't had a chance to do that. What I've been doing is [vocalsound] uh [vocalsound] trying to figure out [disfmarker]
[speaker003:] OK.
[speaker005:] it just seems to me like there's a um [disfmarker] well it seems like there's a bug, because the difference in performance is [disfmarker] it's not gigantic but it's big enough that it [disfmarker] it seems wrong.
[speaker003:] Yeah, I agree,
[speaker005:] and [disfmarker]
[speaker003:] but I thought that the normalization difference was one of the possibilities,
[speaker005:] Yeah, but I don't [disfmarker] I'm not [disfmarker]
[speaker003:] right?
[speaker005:] Yeah, I guess I don't think that the normalization difference is gonna account for everything. So what I was working on is um just going through and checking the headers of the wavefiles,
[speaker003:] OK.
[speaker005:] to see if maybe there was a um [disfmarker] a certain type of compression or something that was done that my script wasn't catching. So that for some subset of the training data, uh the [disfmarker] the [disfmarker] the features I was computing were junk.
[speaker003:] OK.
[speaker005:] Which would you know cause it to perform OK, but uh, you know, the [disfmarker] the models would be all messed up. So I was going through and just double checking that kind of think first, to see if there was just some kind of obvious bug in the way that I was computing the features.
[speaker003:] Mm hmm. I see. OK.
[speaker005:] Looking at all the sampling rates to make sure all the sampling rates were what [disfmarker] eight K, what I was assuming they were,
[speaker003:] Yeah.
[speaker005:] um [disfmarker]
[speaker003:] Yeah, that makes sense, to check all that.
[speaker005:] Yeah. So I was doing that first, before I did these other things, just to make sure there wasn't something [disfmarker]
[speaker003:] Although really, uh uh, a couple three percent uh difference in word error rate uh [comment] could easily come from some difference in normalization, I would think. But
[speaker005:] Yeah, and I think, hhh [disfmarker] [comment] I'm trying to remember but I think I recall that Andreas was saying that he was gonna run sort of the reverse experiment. Uh which is to try to emulate the normalization that we did but with the mel cepstral features. Sort of, you know, back up from the system that he had. I thought he said he was gonna [disfmarker] I have to look back through my [disfmarker] my email from him.
[speaker003:] Yeah, he's probably off at [disfmarker] at uh his meeting now,
[speaker005:] Yeah, he's gone now. Um.
[speaker003:] yeah. Yeah. But yeah the [disfmarker] I sh think they should be [vocalsound] roughly equivalent,
[speaker005:] But [disfmarker]
[speaker003:] um I mean again the Cambridge folk found the PLP actually to be a little better.
[speaker005:] Right.
[speaker003:] Uh So it's [disfmarker] [vocalsound] um I mean the other thing I wonder about was whether there was something just in the [disfmarker] the bootstrapping of their system which was based on [disfmarker] but maybe not, since they [disfmarker]
[speaker005:] Yeah see one thing that's a little bit um [disfmarker] I was looking [disfmarker] I've been studying and going through the logs for the system that um Andreas created. And um his uh [disfmarker] the way that the [disfmarker] [vocalsound] [comment] S R I system looks like it works is that it reads the wavefiles directly, uh and does all of the cepstral computation stuff on the fly.
[speaker003:] Right. Right.
[speaker005:] And, so there's no place where these [disfmarker] where the cepstral files are stored, anywhere that I can go look at and compare to the PLP ones, so whereas with our features, he's actually storing the cepstrum on disk, and he reads those in.
[speaker003:] Right.
[speaker005:] But it looked like he had to give it [disfmarker] uh even though the cepstrum is already computed, he has to give it uh a front end parameter file. Which talks about the kind of uh com computation that his mel cepstrum thing does,
[speaker003:] Uh huh.
[speaker005:] so i I [disfmarker] I don't know if that [disfmarker] it probably doesn't mess it up, it probably just ignores it if it determines that it's already in the right format or something but [disfmarker] the [disfmarker] the [disfmarker] the two processes that happen are a little different. So.
[speaker003:] Yeah. So anyway, there's stuff there to sort out.
[speaker005:] Yeah. Yeah.
[speaker003:] So, OK. Let's go back to what you thought I was asking you.
[speaker005:] Yeah no and I didn't have a chance to do that.
[speaker003:] Ha! Oh! You had the sa same answer anyway.
[speaker005:] Yeah. Yeah. I've been um, [disfmarker] I've been working with um Jeremy on his project and then I've been trying to track down this bug in uh the ICSI front end features.
[speaker003:] Uh huh.
[speaker005:] So one thing that I did notice, yesterday I was studying the um [disfmarker] the uh RASTA code
[speaker003:] Uh huh.
[speaker005:] and it looks like we don't have any way to um control the frequency range that we use in our analysis. We basically [disfmarker] it looks to me like we do the FFT, um and then we just take all the bins and we use everything. We don't have any set of parameters where we can say you know, "only process from you know a hundred and ten hertz to thirty seven fifty".
[speaker003:] Um [disfmarker]
[speaker005:] At least I couldn't see any kind of control for that.
[speaker003:] Yeah, I don't think it's in there, I think it's in the uh uh uh the filters. So, the F F T is on everything, but the filters um, for instance, ignore the [disfmarker] the lowest bins and the highest bins. And what it does is it [disfmarker] it copies
[speaker005:] The [disfmarker] the filters? Which filters?
[speaker003:] um The filter bank which is created by integrating over F F T bins.
[speaker005:] Mm hmm.
[speaker003:] um
[speaker005:] When you get the mel [disfmarker] When you go to the mel scale.
[speaker003:] Right. Yeah, it's bark scale, and it's [disfmarker] it [disfmarker] it um [disfmarker] it actually copies the uh um [disfmarker] the second filters over to the first. So the first filters are always [disfmarker] and you can s you can specify a different number of [vocalsound] uh features [disfmarker] different number of filters, I think, as I recall. So you can specify a different number of filters, and whatever [vocalsound] um uh you specify, the last ones are gonna be ignored. So that [disfmarker] that's a way that you sort of change what the [disfmarker] what the bandwidth is. Y you can't do it without I think changing the number of filters,
[speaker005:] I saw something about uh [disfmarker] that looked like it was doing something like that, but I didn't quite understand it.
[speaker003:] but [disfmarker]
[speaker005:] So maybe [disfmarker]
[speaker003:] Yeah, so the idea is that the very lowest frequencies and [disfmarker] and typically the veriest [comment] highest frequencies are kind of junk.
[speaker005:] Uh huh.
[speaker003:] And so um you just [disfmarker] for continuity you just approximate them by [disfmarker] [vocalsound] by the second to highest and second to lowest.
[speaker005:] Mm hmm.
[speaker003:] It's just a simple thing we put in. And [disfmarker] and so if you h
[speaker005:] But [disfmarker] so the [disfmarker] but that's a fixed uh thing? There's nothing that lets you [disfmarker]
[speaker003:] Yeah, [comment] I think that's a fixed thing. But see [disfmarker] see my point? If you had [disfmarker] [vocalsound] If you had ten filters, [vocalsound] then you would be throwing away a lot at the two ends.
[speaker005:] Mm hmm.
[speaker003:] And if you had [disfmarker] if you had fifty filters, you'd be throwing away hardly anything.
[speaker005:] Mm hmm.
[speaker003:] Um, I don't remember there being an independent way of saying "we're just gonna make them from here to here".
[speaker005:] Use this analysis bandwidth or something.
[speaker003:] But I [disfmarker] I [disfmarker] I don't know, it's actually been awhile since I've looked at it.
[speaker005:] Yeah, I went through the Feacalc code and then looked at you know just calling the RASTA libs [comment] and thing like that. And I didn't [disfmarker] I couldn't see any wh place where that kind of thing was done. But um I didn't quite understand everything that I saw,
[speaker003:] Yeah, see I don't know Feacalc at all.
[speaker005:] so [disfmarker] Mm hmm.
[speaker003:] But it calls RASTA with some options, and um
[speaker005:] Right.
[speaker003:] But I [disfmarker] I think in [disfmarker] I don't know. I guess for some particular database you might find that you could tune that and tweak that to get that a little better, but I think that [vocalsound] in general it's not that critical. I mean there's [disfmarker]
[speaker005:] Yeah.
[speaker003:] You can [disfmarker] You can throw away stuff below a hundred hertz or so and it's just not going to affect phonetic classification at all.
[speaker005:] Another thing I was thinking about was um is there a [disfmarker] I was wondering if there's maybe um [vocalsound] certain settings of the parameters when you compute PLP which would basically cause it to output mel cepstrum. So that, in effect, what I could do is use our code but produce mel cepstrum and compare that directly to [disfmarker]
[speaker003:] Well, it's not precisely. Yeah.
[speaker005:] Hmm.
[speaker003:] I mean, um, [vocalsound] um what you can do is um you can definitely change the [disfmarker] the filter bank from being uh a uh trapezoidal integration to a [disfmarker] a [disfmarker] a triangular one,
[speaker005:] Mm hmm.
[speaker003:] which is what the typical mel [disfmarker] mel cepstral uh filter bank does.
[speaker005:] Mm hmm.
[speaker003:] And some people have claimed that they got some better performance doing that, so you certainly could do that easily. But the fundamental difference, I mean, there's other small differences [disfmarker]
[speaker005:] There's a cubic root that happens, right?
[speaker003:] Yeah, but, you know, as opposed to the log in the other case. I mean [vocalsound] the fundamental d d difference that we've seen any kind of difference from before, which is actually an advantage for the P L P i uh, I think, is that the [disfmarker] the smoothing at the end is auto regressive instead of being cepstral [disfmarker] uh, [comment] from cepstral truncation. So um it's a little more noise robust.
[speaker005:] Hmm.
[speaker003:] Um, and that's [disfmarker] that's why when people started getting databases that had a little more noise in it, like [disfmarker] like uh um Broadcast News and so on, that's why c Cambridge switched to PLP I think.
[speaker005:] Mm hmm.
[speaker003:] So um That's a difference that I don't [vocalsound] think we put any way to get around, since it was an advantage.
[speaker005:] Mm hmm.
[speaker003:] um [vocalsound] uh but we did [disfmarker] eh we did hear this comment from people at some point, that [vocalsound] um it uh they got some better results with the triangular filters rather than the trapezoidal. So that is an option in RASTA.
[speaker005:] Hmm.
[speaker003:] Uh and you can certainly play with that. But I think you're probably doing the right thing to look for bugs first.
[speaker005:] Yeah just [disfmarker] it just seems like this kind of behavior could be caused by you know s some of the training data being messed up.
[speaker003:] I don't know. Could be.
[speaker005:] You know, you're sort of getting most of the way there, but there's a [disfmarker] So I started going through and looking [disfmarker] One of the things that I did notice was that the um log likelihoods coming out of the log recognizer from the PLP data were much lower, much smaller, than for the mel cepstral stuff, and that the average amount of pruning that was happening was therefore a little bit higher for the PLP features.
[speaker003:] Oh huh!
[speaker005:] So, since he used the same exact pruning thresholds for both, I was wondering if it could be that we're getting more pruning.
[speaker003:] Oh! He [disfmarker] he [disfmarker] [vocalsound] He used the identical pruning thresholds even though the s the range of p of the likeli
[speaker005:] Yeah.
[speaker003:] Oh well that's [disfmarker] [vocalsound] That's a pretty good [comment] point right there.
[speaker005:] Right. Right.
[speaker003:] Yeah.
[speaker005:] Yeah, so [disfmarker]
[speaker003:] I would think that you might wanna do something like uh you know, look at a few points to see where you are starting to get significant search errors.
[speaker005:] That's [disfmarker] Right. Well, what I was gonna do is I was gonna take um a couple of the utterances that he had run through, then run them through again but modify the pruning threshold and see if it you know, affects the score.
[speaker003:] Yeah. Yeah.
[speaker005:] So.
[speaker003:] But I mean you could [disfmarker] uh if [disfmarker] if [disfmarker] if that looks promising you could, you know, r uh run [vocalsound] the overall test set with a [disfmarker] with a few different uh pruning thresholds for both,
[speaker005:] Mm hmm. Right.
[speaker003:] and presumably he's running at some pruning threshold that's [disfmarker] that's uh, you know [disfmarker] gets very few search errors but is [disfmarker] is relatively fast
[speaker005:] Mm hmm. Right. I mean, yeah, generally in these things you [disfmarker] you turn back pruning really far,
[speaker003:] and [disfmarker]
[speaker005:] so I [disfmarker] I didn't think it would be that big a deal because I was figuring well you have it turned back so far that you know it [disfmarker]
[speaker003:] But you may be in the wrong range for the P L P features for some reason.
[speaker005:] Yeah. Yeah. Yeah. And the uh the [disfmarker] the run time of the recognizer on the PLP features is longer which sort of implies that the networks are bushier, you know, there's more things it's considering which goes along with the fact that the matches aren't as good. So uh, you know, it could be that we're just pruning too much. So.
[speaker003:] Yeah. Yeah, maybe just be different kind of distributions and [disfmarker] and
[speaker005:] Mm hmm.
[speaker003:] yeah so that's another possible thing.
[speaker005:] Mm hmm.
[speaker003:] They [disfmarker] they should [disfmarker] really shouldn't [disfmarker] There's no particular reason why they would be exactly [disfmarker] behave exactly the same.
[speaker005:] Mm hmm. Right. Right.
[speaker003:] So.
[speaker005:] So. There's lots of little differences. So.
[speaker003:] Yeah.
[speaker005:] Uh.
[speaker003:] Yeah.
[speaker005:] Trying to track it down.
[speaker003:] Yeah. I guess this was a little bit off topic, I guess,
[speaker005:] Yeah
[speaker003:] because I was [disfmarker] I was thinking in terms of th this as being a [disfmarker] a [disfmarker] a [disfmarker] a core [vocalsound] item that once we [disfmarker] once we had it going we would use for a number of the front end things also.
[speaker005:] Mm hmm.
[speaker003:] So. um Wanna [disfmarker]
[speaker002:] That's [disfmarker] as far as my stuff goes,
[speaker003:] What's [disfmarker] what's on [disfmarker]
[speaker002:] yeah,
[speaker003:] Yeah.
[speaker002:] well I [vocalsound] tried this mean subtraction method. Um. Due to Avendano, [vocalsound] I'm taking s um [vocalsound] six seconds of speech, um [vocalsound] I'm using two second [vocalsound] FFT analysis frames, [vocalsound] stepped by a half second so it's a quarter length step and I [disfmarker] [vocalsound] I take that frame and four f the four [disfmarker] I take [disfmarker] Sorry, I take the current frame and the four past frames and the [vocalsound] four future frames and that adds up to six seconds of speech. And I calculate um [vocalsound] the spectral mean, [vocalsound] of the log magnitude spectrum [pause] over that N. I use that to normalize the s the current center frame [vocalsound] by mean subtraction. And I then [disfmarker] then I move to the next frame and I [disfmarker] [vocalsound] I do it again. Well, actually I calculate all the means first and then I do the subtraction. And um [vocalsound] the [disfmarker] I tried that with HDK, the Aurora setup of HDK training on clean TI digits, and um [vocalsound] it [disfmarker] it helped um in a phony reverberation case um [vocalsound] where I just used the simulated impulse response um [vocalsound] the error rate went from something like eighty it was from something like eighteen percent [vocalsound] to um four percent. And on meeting rec recorder far mike digits, mike [disfmarker] on channel F, it went from um [vocalsound] [vocalsound] forty one percent error to eight percent error.
[speaker005:] On [disfmarker] on the real data, not with artificial reverb?
[speaker002:] Right.
[speaker005:] Uh huh.
[speaker002:] And that [disfmarker] that was um [vocalsound] trained on clean speech only, which I'm guessing is the reason why the baseline was so bad. And [disfmarker]
[speaker003:] That's ac actually a little side point is I think that's the first results that we have uh uh uh of any sort on the far field uh [disfmarker] on [disfmarker] on the far field data uh for [disfmarker] recorded in [disfmarker] in meetings.
[speaker002:] Oh um actually um Adam ran the SRI recognizer.
[speaker003:] Did he? On the near field, on the ne
[speaker002:] On the far field also. He did one PZM channel and one PDA channel.
[speaker003:] Oh did he? Oh! I didn't recall that. What kind of numbers was he getting with that?
[speaker002:] I [disfmarker] [vocalsound] I'm not sure, I think it was about five percent error for the PZM channel.
[speaker003:] Five.
[speaker002:] f I think. Yeah.
[speaker003:] So why were you getting forty one here? Is this [disfmarker]
[speaker002:] Um. I [disfmarker] I'm g I'm guessing it was the [disfmarker] the training data. Uh, clean TI digits is, like, pretty pristine [vocalsound] training data, and if they trained [vocalsound] the SRI system on this TV broadcast type stuff, I think it's a much wider range of channels and it [disfmarker]
[speaker003:] No, but wait a minute. I [disfmarker] I [disfmarker] I th [disfmarker] I think he [disfmarker] What am I saying here? Yeah, so that was the SRI system. Maybe you're right. Yeah. Cuz it was getting like one percent [disfmarker] [vocalsound] So it's still this kind of ratio. It was [disfmarker] it was getting one percent or something on the near field. Wasn't it?
[speaker005:] Mm hmm, or it wa a it was around one.
[speaker003:] Yeah. Yeah. I think it was getting around one percent for the near [disfmarker] for the n for the close mike.
[speaker005:] Yeah.
[speaker002:] Huh?
[speaker003:] So it was like one to five [disfmarker]
[speaker002:] OK.
[speaker003:] So it's still this kind of ratio. It's just [disfmarker] yeah, it's a lot more training data. So So probably it should be something we should try then is to [disfmarker] is to see if [disfmarker] is [vocalsound] at some point just to take [disfmarker] i to transform the data and then [disfmarker] [vocalsound] and then uh use th use it for the SRI system.
[speaker002:] b You me you mean um ta
[speaker003:] So you're [disfmarker] so you have a system which for one reason or another is relatively poor,
[speaker002:] Yeah.
[speaker003:] and [disfmarker] and uh you have something like forty one percent error uh and then you transform it to eight by doing [disfmarker] doing this [disfmarker] this work. Um. So here's this other system, which is a lot better, but there's still this kind of ratio. It's something like five percent error [vocalsound] with the [disfmarker] the distant mike, and one percent with the close mike.
[speaker002:] OK.
[speaker003:] So the question is [vocalsound] how close to that one can you get [vocalsound] if you transform the data using that system.
[speaker002:] r Right, so [disfmarker] so I guess this SRI system is trained on a lot of s Broadcast News or Switchboard data.
[speaker003:] Yeah.
[speaker002:] Is that right? Do you know which one it is?
[speaker005:] It's trained on a lot of different things. Um. It's trained on uh a lot of Switchboard, Call Home,
[speaker002:] Uh huh.
[speaker005:] um a bunch of different sources, some digits, there's some digits training in there.
[speaker002:] OK. O one thing I'm wondering about is what this mean subtraction method [vocalsound] um will do if it's faced with additive noise.
[speaker001:] Hmm.
[speaker002:] Cuz I [disfmarker] I [disfmarker] it's cuz I don't know what log magnitude spectral subtraction is gonna do to additive noise.
[speaker003:] Yeah, well, it's [disfmarker] it's not exactly the right thing
[speaker002:] That's [disfmarker] that's the [disfmarker] Uh huh.
[speaker003:] but [vocalsound] uh [vocalsound] but you've already seen that cuz there is added noise here.
[speaker002:] That's [disfmarker] that's [disfmarker] Yeah, that's true.
[speaker003:] Yeah.
[speaker002:] That's a good point.
[speaker003:] So um [disfmarker]
[speaker002:] OK, so it's then [disfmarker] then it's [disfmarker] it's [disfmarker] it's reasonable to expect it would be helpful if we used it with the SRI system and
[speaker003:] Yeah, I mean, as helpful [disfmarker] I mean, so that's the question. Yeah, w we're often asked this when we work with a system that [disfmarker] that isn't [disfmarker] isn't sort of industry [disfmarker] industry standard great,
[speaker002:] Uh huh.
[speaker003:] uh and we see some reduction in error using some clever method, then, you know, will it work on a [disfmarker] [vocalsound] on a [disfmarker] on a good system. So uh you know, this other one's [disfmarker] it was a pretty good system. I think, you know, one [disfmarker] one percent word error rate on digits is [disfmarker] uh digit strings is not [vocalsound] uh you know stellar, but [disfmarker] but given that this is real [vocalsound] digits, as opposed to uh sort of laboratory [disfmarker]
[speaker002:] Mm hmm.
[speaker003:] Well.
[speaker005:] And it wasn't trained on this task either.
[speaker003:] And it wasn't trained on this task. Actually one percent is sort of [disfmarker] you know, sort of in a reasonable range. People would say "yeah, I could [disfmarker] I can imagine getting that".
[speaker002:] Mm hmm.
[speaker003:] And uh so the [disfmarker] the four or five percent or something is [disfmarker] is [disfmarker] is quite poor.
[speaker002:] Mm hmm.
[speaker003:] Uh, you know, if you're doing a uh [disfmarker] [vocalsound] a sixteen digit uh credit card number you'll basically get it wrong almost all the time. So. So. Uh, [vocalsound] um
[speaker002:] Hmm.
[speaker003:] a significant reduction in the error for that would be great.
[speaker002:] Huh, OK.
[speaker003:] And [disfmarker] and then, uh Yeah. So. Yeah. Cool.
[speaker002:] Sounds good.
[speaker003:] Yeah. Alright, um, I actually have to run. So I don't think I can do the digits, but um, [vocalsound] I guess I'll leave my microphone on?
[speaker005:] Uh, yeah.
[speaker003:] Yeah. Thank you.
[speaker005:] Yep. Yeah. That'll work.
[speaker003:] I can be out of here quickly. [comment] [comment] [vocalsound] [vocalsound] That's I just have to run for another appointment. OK, I t Yeah. I left it on. OK.