[speaker005:] OK, we're on.
[speaker002:] OK.
[speaker005:] So, I mean, everyone who's on the wireless check that they're on.
[speaker006:] C we [disfmarker]
[speaker007:] Alright.
[speaker003:] I see.
[speaker006:] Yeah.
[speaker003:] Yeah.
[speaker005:] OK, our agenda was quite short.
[speaker002:] Oh, could you [pause] close the door, maybe?
[speaker005:] Sure.
[speaker002:] Yeah.
[speaker005:] Two items, which was, uh, digits and possibly stuff on [disfmarker] on, uh, forced alignment, which Jane said that Liz and Andreas had in information on,
[speaker006:] Mm hmm.
[speaker005:] but they didn't, so.
[speaker002:] I guess the only other thing, uh, for which I [disfmarker]
[speaker006:] We should do that second, because Liz might join us in time for that.
[speaker005:] OK.
[speaker002:] Um. OK, so there's digits, alignments, and, um, I guess the other thing, [vocalsound] which I came unprepared for, uh, [vocalsound] is, uh, to dis s s see if there's anything anybody wants to discuss about the Saturday meeting.
[speaker005:] Right.
[speaker002:] So. Any [disfmarker] I mean, maybe not.
[speaker005:] Digits and alignments. But [disfmarker]
[speaker002:] Uh.
[speaker006:] Talk about aligning people's schedules.
[speaker002:] Yeah.
[speaker005:] Yeah.
[speaker003:] Mm hmm.
[speaker002:] Yeah. I mean [disfmarker] Right. Yeah, I mean, it was [disfmarker]
[speaker005:] Yeah, it's forced alignment of people's schedules.
[speaker006:] Yeah. If we're very [disfmarker]
[speaker004:] Forced align.
[speaker002:] Yeah.
[speaker006:] Yeah.
[speaker002:] With [disfmarker] with [disfmarker] whatever it was, a month and a half or something ahead of time, the only time we could find in common [disfmarker] roughly in common, was on a Saturday.
[speaker004:] Yeah.
[speaker005:] Yep.
[speaker002:] Ugh.
[speaker006:] It's pretty sad.
[speaker002:] Yeah.
[speaker006:] Yeah.
[speaker003:] Have [disfmarker] Have we thought about having a conference call to include him in more of [disfmarker] [vocalsound] in more of the meeting? I [disfmarker] I mean, I don't know, if we had the [disfmarker] if we had the telephone on the table [disfmarker]
[speaker002:] No. But, h I mean, he probably has to go do something.
[speaker006:] No, actually I [disfmarker] I have to [disfmarker] I have to shuttle [pause] kids from various places to various other places.
[speaker002:] Right?
[speaker003:] I see. OK.
[speaker006:] So.
[speaker002:] Yeah.
[speaker006:] And I don't have [disfmarker] and I don't, um, have a cell phone
[speaker004:] A cell phone?
[speaker006:] so I can't be having a conference call while driving.
[speaker003:] No. [comment] It's not good.
[speaker002:] R r right.
[speaker003:] That's not good.
[speaker002:] So we have to [disfmarker] we [disfmarker]
[speaker006:] Plus, it would make for interesting noise [disfmarker] background noise.
[speaker005:] Yep.
[speaker006:] Uh [disfmarker]
[speaker002:] So we have to equip him with a [disfmarker] with a [disfmarker] [vocalsound] with a head mounted, uh, cell phone
[speaker005:] Ye we and we'd have to force you to read lots and lots of digits,
[speaker002:] and [disfmarker]
[speaker006:] Oh, yeah.
[speaker005:] so it could get real [disfmarker] [vocalsound] real car noise.
[speaker004:] Yeah.
[speaker006:] Oh, yeah.
[speaker007:] Take advantage.
[speaker004:] And with the kids in the background.
[speaker006:] I'll let [disfmarker] I'd let [disfmarker]
[speaker004:] Yeah.
[speaker006:] I let, uh, my five year old have a try at the digits, eh.
[speaker002:] Yeah.
[speaker005:] So, anyway, I can talk about digits. Um, did everyone get the results or shall I go over them again? I mean that it was basically [disfmarker] the only thing that was even slightly surprising was that the lapel did so well. Um, and in retrospect that's not as surprising as maybe i it shouldn't have been as surprising as I [disfmarker] as [disfmarker] as I felt it was. The lapel mike is a very high quality microphone. And as Morgan pointed out, that there are actually some advantages to it in terms of breath noises and clothes rustling [pause] if no one else is talking.
[speaker006:] Exactly.
[speaker004:] Yeah.
[speaker007:] Mm hmm.
[speaker005:] Um, so, uh [disfmarker]
[speaker007:] It's g it [disfmarker]
[speaker002:] Well, it's [disfmarker] Yeah, sort of the bre the breath noises and the mouth clicks and so forth like that, the lapel's gonna be better on.
[speaker004:] Or the cross talk. Yeah.
[speaker002:] The lapel is typically worse on the [disfmarker] on clothes rustling, but if no one's rustling their clothes,
[speaker005:] Right. I mean, a lot of people are just sort of leaning over and reading the digits,
[speaker002:] it's [disfmarker] it's [disfmarker]
[speaker005:] so it's [disfmarker] it's a very different task than sort of the natural.
[speaker004:] Yeah. You don't move much during reading digits, I think.
[speaker005:] So.
[speaker002:] Yeah. Yeah.
[speaker005:] Right.
[speaker007:] Probably the fact that it picks up other people's speakers [disfmarker] other people's talking is an indication of that it [disfmarker] the fact it is a good microphone.
[speaker004:] Yeah.
[speaker005:] Right.
[speaker002:] Right. So in the digits, in most [disfmarker] most cases, there weren't other people talking.
[speaker007:] So.
[speaker005:] Right.
[speaker006:] D do the lapel mikes have any directionality to them?
[speaker002:] So. There typically don't, no.
[speaker006:] Because I [disfmarker] I suppose you could make some that have sort of [disfmarker] that you have to orient towards your mouth, and then it would [disfmarker]
[speaker005:] They have a little bit, but they're not noise cancelling. So, uh [disfmarker]
[speaker002:] They're [disfmarker] they're intended to be omni directional.
[speaker005:] Right.
[speaker006:] Mm hmm.
[speaker002:] And th it's [disfmarker] and because you don't know how people are gonna put them on, you know.
[speaker005:] Right. So, also, Andreas, on that one the [disfmarker] the back part of it should be right against your head. And that will he keep it from flopping aro up and down as much.
[speaker006:] It is against my head.
[speaker005:] OK.
[speaker002:] Yeah. Um. Yeah, we actually talked about this in the, uh, front end meeting this morning, too.
[speaker005:] Uh huh.
[speaker002:] Much the same thing, and [disfmarker] and it was [disfmarker] uh, I mean, there the point of interest to the group was primarily that, um, [vocalsound] the, uh [disfmarker] the system that we had that was based on H T K, that's used by, you know, [pause] all the participants in Aurora, [vocalsound] was so much worse [vocalsound] than the [disfmarker] than the S R
[speaker005:] Everybody.
[speaker002:] And the interesting thing is that even though, [vocalsound] yes, it's a digits task and that's a relatively small number of words and there's a bunch of digits that you train on, [vocalsound] it's just not as good as having a [disfmarker] a l very large amount of data and training up a [disfmarker] a [disfmarker] a nice good big [vocalsound] HMM. Um, also you had the adaptation in the SRI system, which we didn't have in this. Um. So. Um.
[speaker006:] And we know [disfmarker] Di did I send you some results without adaptation?
[speaker005:] No. Or if you did, I didn't include them,
[speaker002:] I s I think Stephane, uh, had seen them.
[speaker005:] cuz it was [disfmarker]
[speaker002:] So [disfmarker]
[speaker006:] Yeah, I think I did, actually. So there was a significant loss from not doing the adaptation.
[speaker002:] Yeah.
[speaker006:] Um. A [disfmarker] a [disfmarker] a couple percent or some I mean [disfmarker] Well, I don't know it [disfmarker] Overall [disfmarker] Uh, I [disfmarker] I don't remember, but there was [disfmarker] [nonvocalsound] there was a significant, um, loss or win [comment] from adaptation [disfmarker] with [disfmarker] with adaptation. And, um, that was the phone loop adaptation. And then there was a very small [disfmarker] like point one percent on the natives [disfmarker] uh, win from doing, um, you know, adaptation to [pause] the recognition hypotheses. And [pause] I tried both means adaptation and means and variances, and the variances added another [disfmarker] or subtracted another point one percent. So, [vocalsound] it's, um [disfmarker] that's the number there. Point six, I believe, is what you get with both, uh, means and variance adaptation.
[speaker005:] Right.
[speaker002:] But I think one thing is that, uh, I would presume [disfmarker] Hav Have you ever t [vocalsound] Have you ever tried this exact same recognizer out on the actual TI digits test set?
[speaker006:] This exact same recognizer? No.
[speaker002:] It might be interesting to do that. Cuz my [disfmarker] my [disfmarker] cuz my sense, um [disfmarker]
[speaker006:] But [disfmarker] but, I have [disfmarker] I mean, people [disfmarker] people at SRI are actually working on digits.
[speaker005:] I bet it would do even slightly better.
[speaker006:] I could [disfmarker] and they are using a system that's, um [disfmarker] you know, h is actually trained on digits, um, but h h otherwise uses the same, you know, decoder, the same, uh, training methods, and so forth,
[speaker002:] Mm hmm.
[speaker006:] and I could ask them what they get [pause] on TI digits.
[speaker002:] Yeah, bu although I'd be [disfmarker] I think it'd be interesting to just take this exact actual system so that these numbers were comparable
[speaker006:] Mm hmm. Well, Adam knows how to run it,
[speaker002:] and try it out on TI digits.
[speaker005:] Yeah. No problem.
[speaker006:] so you just make a f
[speaker002:] Yeah. Yeah. Yeah. Cuz our sense from the other [disfmarker] from the Aurora, uh, task is that [disfmarker]
[speaker005:] And try it with TI digits?
[speaker006:] Mm hmm.
[speaker002:] I mean, cuz we were getting sub one percent [vocalsound] numbers on TI digits also with the tandem thing.
[speaker006:] Mm hmm. Mmm.
[speaker002:] So, [vocalsound] one [disfmarker] so there were a number of things we noted from this. One is, yeah, the SRI system is a lot better than the HTK [disfmarker]
[speaker006:] Hmm.
[speaker002:] this, you know, very limited training HTK system.
[speaker006:] Mm hmm.
[speaker002:] Uh, but the other is that, um, the digits [vocalsound] recorded here in this room with these close mikes, i uh, are actually a lot harder than the [pause] studio recording TI digits.
[speaker006:] Mm hmm.
[speaker002:] I think, you know, one reason for that, uh, might be that there's still [disfmarker] even though it's close talking, there still is some noise and some room acoustics.
[speaker006:] Mm hmm.
[speaker002:] And another might be that, uh, I'd [disfmarker] I would presume that in the studio, uh, uh, situation recording read speech that if somebody did something a little funny or n pronounced something a little funny or made a little [disfmarker] that they didn't include it,
[speaker005:] They didn't include it.
[speaker002:] they made them do it again.
[speaker005:] Whereas, I took out [pause] the ones that I noticed that were blatant [disfmarker] that were correctable.
[speaker002:] Mmm. Yeah.
[speaker005:] So that, if someone just read the wrong digit, I corrected it. And then there was another one where Jose couldn't tell whether [disfmarker] I couldn't tell whether he was saying zero or six.
[speaker002:] Yeah.
[speaker005:] And I asked him and he couldn't tell either.
[speaker009:] Hmm.
[speaker005:] So I just cut it out. You know, so I just e edited out the first, i uh, word of the utterance.
[speaker002:] Yeah.
[speaker005:] Um, so there's a little bit of correction but it's definitely not as clean as TI digits. So my expectations is TI digits would, especially [disfmarker] I think TI digits is all [pause] American English. Right? So it would probably do even a little better still
[speaker002:] Mm hmm.
[speaker005:] on the SRI system, but we could give it a try.
[speaker006:] Well. But [pause] remember, we're using a telephone bandwidth front end here, uh, on this, uh [disfmarker] on this SRI system, so, [vocalsound] um, I was [disfmarker] I thought that maybe that's actually a good thing because it [disfmarker] it gets rid of some of the [disfmarker] uh, the noises, um, you know, in the [disfmarker] the [disfmarker] below and above the [disfmarker] um, the, you know, speech bandwidth
[speaker002:] Mm hmm. Mm hmm.
[speaker006:] and, um, I suspect that to get sort of the last bit out of these higher quality recordings you would have to in fact, uh, use models that, uh, were trained on wider band data. And of course we can't do that or [disfmarker]
[speaker005:] Wha what's TI digits? I thought t
[speaker002:] It's wide band, yeah. It's [disfmarker] in [disfmarker] in fact, we looked it up
[speaker005:] It is wide band. OK.
[speaker002:] and it was actually twenty kilohertz sampling.
[speaker005:] Oh, that's right. I [disfmarker] I did look that up.
[speaker006:] Mm hmm.
[speaker005:] I couldn't remember whether that was TI digits or one of the other digit tasks.
[speaker002:] Yeah.
[speaker006:] Right. But [disfmarker] but, I would [disfmarker] Yeah. It's [disfmarker] it's easy enough to try, just run it on [disfmarker]
[speaker005:] Mm hmm.
[speaker002:] Yeah.
[speaker005:] So, Morgan, you're getting a little breath noise.
[speaker002:] See w
[speaker006:] Now, eh, does [disfmarker] one [disfmarker] one issue [disfmarker]
[speaker005:] You might wanna move the mike down a little bit.
[speaker006:] one issue with [disfmarker] with that is that [vocalsound] um, the system has this, uh, notion of a speaker to [disfmarker] which is used in adaptation, variance norm uh, you know, both in, uh, mean and variance normalization and also in the VTL [pause] estimation.
[speaker002:] Mm hmm.
[speaker006:] So [disfmarker]
[speaker005:] Yeah, I noticed the script that extracted it.
[speaker006:] Do y? Is [disfmarker]? So does [disfmarker] so th so does [disfmarker] does, um, [vocalsound] the TI digits database have speakers that are known?
[speaker005:] Yep. Yep.
[speaker006:] And is there [disfmarker] is there enough data or a comparable [disfmarker] comparable amount of data to [disfmarker] to what we have in our recordings here?
[speaker005:] That I don't know. I don't know. I don't know how many speakers there are, and [disfmarker] and how many speakers per utterance.
[speaker006:] OK.
[speaker002:] Yeah. Well, the other thing would be to do it without the adaptation and compare to these numbers without the adaptation. That would [disfmarker]
[speaker006:] Right. Uh, but I'm not so much worried about the adaptation, actually, than [disfmarker] than the, um, [vocalsound] um [disfmarker] the, uh, VTL estimation.
[speaker005:] Right.
[speaker006:] If you have only one utterance per speaker you might actually screw up on estimating the [disfmarker] the warping, uh, factor. So, um [disfmarker]
[speaker005:] I strongly suspect that they have more speakers than we do.
[speaker006:] Right. But it's not the amount of speakers,
[speaker005:] So, uh [disfmarker]
[speaker006:] it's the num it's the amount of data per speaker.
[speaker005:] Right. So we [disfmarker] we could probably do an extraction that was roughly equivalent.
[speaker006:] Right. Right.
[speaker005:] Um.
[speaker006:] So [disfmarker]
[speaker005:] So, although I [disfmarker] I sort of know how to run it, there are a little [disfmarker] a f few details here and there that I'll have to [pause] dig out.
[speaker006:] OK. The key [disfmarker] So th the system actually extracts the speaker ID from the waveform names.
[speaker005:] Right. I saw that.
[speaker006:] And there's a [disfmarker] there's a script [disfmarker] and that is actually all in one script. So there's this one script that parses waveform names and extracts things like the, um, speaker, uh, ID or something that can stand in as a speaker ID. So, we might have to modify that script to recognize the, um, speakers, [vocalsound] um, in the [disfmarker] in the, uh, um, [vocalsound] TI digits [pause] database.
[speaker005:] Right. Right. And that, uh [disfmarker]
[speaker006:] Or you can fake [disfmarker] you can fake [pause] names for these waveforms that resemble the names that we use here for the [disfmarker] for the meetings.
[speaker005:] Right.
[speaker006:] That would be the, sort of [disfmarker] probably the safest way to do [disfmarker]
[speaker005:] I might have to do that anyway to [disfmarker] to do [disfmarker] because we may have to do an extract to get the [pause] amount of data per speaker about right.
[speaker006:] Uh huh.
[speaker005:] The other thing is, isn't TI digits isolated digits?
[speaker006:] Right.
[speaker005:] Or is that another one? I'm [disfmarker] I looked through a bunch of the digits t corp corpora, and now they're all blurring.
[speaker002:] Mm hmm.
[speaker005:] Cuz one of them was literally people reading a single digit. And then others were connected digits.
[speaker002:] Yeah. Most of TI digits is connected digits, I think.
[speaker005:] OK.
[speaker002:] The [disfmarker] I mean, we had a Bellcore corpus that we were using.
[speaker005:] Maybe it's the Bell Gram.
[speaker002:] It was [disfmarker] [vocalsound] that's [disfmarker] that was isolated digits.
[speaker005:] Bell Digits. Alright.
[speaker006:] By the way, I think we can improve these numbers if we care to compr improve them [vocalsound] by, um, [vocalsound] not starting with the Switchboard models but by taking the Switchboard models and doing supervised adaptation on a small amount of digit data collected in this setting.
[speaker002:] Um.
[speaker005:] Yep.
[speaker006:] Because that would adapt your models to the room acoustics and f for the far field microphones, you know, to the noise. And that should really improve things, um, further. And then you use those adapted models, which are not speaker adapted but sort of acous you know, channel adapted [disfmarker]
[speaker005:] Channel adapted.
[speaker006:] use that as the starting models for your speaker adaptation.
[speaker002:] Yeah. [vocalsound] But the thing is, uh [disfmarker] I mean, w when you [disfmarker] it depends whether you're ju were just using this as a [disfmarker] [vocalsound] a starter task for [disfmarker] you know, to get things going for conversational or if we're really interested i in connected digits. And I [disfmarker] I think the answer is both.
[speaker006:] Well, I don't know.
[speaker002:] And for [disfmarker] for connected digits over the telephone you don't actually want to put a whole lot of effort into adaptation because [vocalsound] somebody [pause] gets on the phone and says a number and then you just want it. You don't [disfmarker] don't, uh [disfmarker]
[speaker003:] This is [disfmarker] this [disfmarker] that one's better.
[speaker006:] Right.
[speaker003:] Mm hmm.
[speaker006:] Um, but, you know, I [disfmarker] uh, my impression was that you were actually interested in the far field microphone, uh, problem, I mean. So, you want to [disfmarker] you want to [disfmarker] That's the obvious thing to try.
[speaker003:] Oh. Oh.
[speaker006:] Right? Then, eh [disfmarker] because you [disfmarker] you don't have any [disfmarker]
[speaker002:] Right.
[speaker003:] Yeah.
[speaker006:] That's where the most m acoustic mismatch is between the currently used models and the [disfmarker] the r the set up here.
[speaker002:] Right.
[speaker006:] So.
[speaker002:] Yeah. So that'd be anoth another interesting data point.
[speaker006:] Mm hmm.
[speaker002:] I mean, I [disfmarker] I guess I'm saying I don't know if we'd want to do that as the [disfmarker] as [disfmarker]
[speaker005:] Other way.
[speaker004:] Other way.
[speaker005:] Liz [disfmarker]
[speaker001:] Now you're all watching me.
[speaker005:] It f it clips over your ears.
[speaker001:] Alright. This way.
[speaker005:] There you go.
[speaker003:] If you have a strong fe if you have a strong preference, you could use this.
[speaker001:] You're all watching. This is terrible.
[speaker003:] It's just we [disfmarker] we think it has some spikes. So, uh, we [disfmarker] we didn't use that one.
[speaker001:] I'll get it.
[speaker003:] But you could if you want.
[speaker002:] Yeah. At any rate, I don't know if w
[speaker003:] I don't know. And Andre Andreas, your [disfmarker] your microphone's a little bit low.
[speaker006:] It is?
[speaker002:] Yeah.
[speaker003:] Yeah.
[speaker002:] I don't know if we wanna use that as the [disfmarker]
[speaker006:] Uh.
[speaker005:] Uh, it pivots.
[speaker003:] So if you see the picture
[speaker006:] I I [disfmarker]
[speaker005:] It [disfmarker] it [disfmarker] like this.
[speaker003:] and then you have to scr
[speaker006:] I [disfmarker] I already adjusted this a number of times.
[speaker005:] Eh.
[speaker006:] I [disfmarker] I
[speaker005:] Yeah, I think these mikes are not working as well as I would like.
[speaker006:] can't quite seem to [disfmarker] Yeah, I think this contraption around your head is not [pause] working so well.
[speaker002:] Too many adju too many adjustments. Yeah. Anyway, what I was saying is that I [disfmarker] I think I probably wouldn't want to see that as sort of like the norm, that we compared all things to.
[speaker003:] That looks good. Yeah.
[speaker002:] To, uh, the [disfmarker] to have [disfmarker] have all this ad all this, uh, adaptation. But I think it's an important data point, if you're [disfmarker] if [disfmarker]
[speaker006:] Right.
[speaker002:] Yeah. Um. The other thing that [disfmarker] that, uh [disfmarker] of course, what Barry was looking at was [disfmarker] was just that, the near versus far. And, yeah, the adaptation would get [vocalsound] th some of that.
[speaker006:] Mm hmm.
[speaker002:] But, I think even [disfmarker] even if there was, uh, only a factor of two or something, like I was saying in the email, I think that's [disfmarker] [vocalsound] that's a big factor.
[speaker006:] Mm hmm.
[speaker002:] So [disfmarker]
[speaker005:] Liz, you could also just use the other mike if you're having problems with that one.
[speaker002:] N
[speaker003:] Well.
[speaker001:] OK.
[speaker003:] Yeah. This would be OK. We [disfmarker] we [disfmarker] we think that this has spikes on it,
[speaker001:] It's this thing's [disfmarker] This is too big for my head.
[speaker003:] so it's not as good acoustically,
[speaker006:] Yeah, basically your ears are too big.
[speaker003:] but [disfmarker]
[speaker006:] I mean, mine are too. E th everybody's ears are too big for these things.
[speaker001:] No, my [disfmarker] my [disfmarker] But this is too big for my head.
[speaker006:] Uh [disfmarker]
[speaker001:] So, I mean, [comment] [comment] it doesn't [disfmarker] you know, it's sit
[speaker003:] Well, if you'd rather have this one then it's [disfmarker]
[speaker001:] OK.
[speaker002:] Yeah.
[speaker005:] Oh, well.
[speaker002:] It's [pause] great.
[speaker005:] So the [disfmarker] To get that, uh, pivoted this way, it pivots like this.
[speaker001:] No this way.
[speaker005:] Yeah.
[speaker001:] Yeah.
[speaker005:] There you go.
[speaker003:] And there's a screw that you can tighten.
[speaker005:] And then it [disfmarker] Right.
[speaker001:] Right. I already [pause] tried to get it close.
[speaker003:] Good.
[speaker005:] So if it doesn't bounce around too much, that's actually good placement.
[speaker003:] That looks good.
[speaker001:] OK.
[speaker005:] But it looks like it's gonna bounce a lot.
[speaker002:] So, where were we? Uh [disfmarker] [vocalsound] Yeah.
[speaker003:] Yeah.
[speaker005:] Digits. Adaptation.
[speaker002:] Uh, adaptation, non adaptation, um, factor of two, um [disfmarker]
[speaker006:] What k u By the way, wh what factor of two did you [disfmarker]?
[speaker002:] Oh, yeah. I know what I was go w
[speaker006:] I mean [disfmarker]
[speaker002:] Oh, no, no. It's tha that [disfmarker] that we were saying, you know, well is [disfmarker] how much worse is far than near, you know.
[speaker006:] Oh, th OK.
[speaker002:] And I mean it depends on which one you're looking at,
[speaker006:] That factor of two. Mm hmm.
[speaker002:] but for the everybody, it's [vocalsound] little under a factor or two. Yeah. I [disfmarker] I know what I was thinking was that maybe, uh, i i we could actually t t try at least looking at, uh, some of the [disfmarker] the large vocabulary speech from a far microphone, at least from the good one.
[speaker006:] Mm hmm.
[speaker002:] I mean, before I thought we'd get, you know, a hundred and fifty percent error or something,
[speaker006:] Mm hmm.
[speaker002:] but if [disfmarker] [vocalsound] if, uh [disfmarker] if we're getting thirty five, forty percent or something, [vocalsound] u um [disfmarker]
[speaker001:] Actually if you run, though, on a close talking mike over the whole meeting, during all those silences, you get, like, four hundred percent word error.
[speaker002:] Mm hmm. Right. I understand. But doing the same kind of limited thing [disfmarker]
[speaker001:] Or [disfmarker] or some high number.
[speaker002:] Yeah, sure. Get all these insertions. But I'm saying if you do the same kind of limited thing [vocalsound] as people have done in Switchboard evaluations or as [disfmarker] a
[speaker001:] Yeah. Where you know who the speaker is and there's no overlap?
[speaker002:] Yeah.
[speaker001:] And you do just the far field for those regions?
[speaker002:] Yeah. The same sort of numbers that we got those graphs from.
[speaker005:] Could we do exactly the same thing that we're doing now, but do it with a far field mike?
[speaker002:] Right? Yeah, do it with one of [disfmarker] on
[speaker005:] Cuz we extract the times from the near field mike, but you use the acoustics from the far field mike.
[speaker001:] Right. I understand that. I just meant that [disfmarker] so you have [pause] three choices. There's, um [disfmarker] You can use times where that person is talking only from the transcripts but the segmentations were [disfmarker] were synchronized. Or you can do a forced alignment on the close talking to determine that, the you know, within this segment, these really were the times that this person was talking and elsewhere in the segment other people are overlapping and just front end those pieces. Or you can run it on the whole data, which is [disfmarker] which is, you know, a [disfmarker]
[speaker002:] But [disfmarker] but [disfmarker] but how did we get the [disfmarker] how did we determine the links, uh, that we're testing on in the stuff we reported?
[speaker001:] In the H L T paper we took [pause] segments that are channel [disfmarker] time aligned, which is now h being changed in the transcription process, which is good, and we took cases where the transcribers said there was only one person talking here, because no one else had time [disfmarker] any words in that segment and called that "non overlap".
[speaker002:] And tha And that's what we were getting those numbers from.
[speaker001:] Yes.
[speaker002:] Right.
[speaker001:] Tho good [disfmarker] the good numbers. The bad numbers were from [pause] the segments where there was overlap.
[speaker002:] Well, we could start with the good ones. But anyway [disfmarker] so I think that we should try it once with [vocalsound] the same conditions that were used to create those,
[speaker001:] Yeah.
[speaker002:] and in those same segments just use one of the P Z
[speaker001:] Right. So we [disfmarker] we can do that.
[speaker002:] And then, you know, I mean, the thing is if we were getting, uh [disfmarker] what, thirty five, forty percent, something like that on [disfmarker] on that particular set,
[speaker001:] Yeah.
[speaker002:] uh, does it go to seventy or eighty? Or, does it use up so much memory we can't decode it?
[speaker001:] Right. It might also depend on which speaker th it is and how close they are to the PZM?
[speaker002:] Uh [disfmarker]
[speaker001:] I don't know how different they are from each other.
[speaker006:] You want to probably choose the PZM channel that is closest to the speaker.
[speaker001:] To be best [disfmarker]
[speaker005:] For this particular digit ones, I just picked that one.
[speaker004:] Yeah.
[speaker001:] f
[speaker002:] Well [disfmarker]
[speaker001:] OK.
[speaker006:] Oh, OK.
[speaker001:] So we would then use that one, too,
[speaker005:] So [disfmarker]
[speaker002:] This is kind of central.
[speaker001:] or [disfmarker]?
[speaker002:] You know, it's [disfmarker] so i but I would [disfmarker] I'd pick that one. It'll be less good for some people than for other, but I [disfmarker] I'd like to see it on the same [disfmarker] exact same data set that [disfmarker] that we did the other thing on.
[speaker005:] Actually [disfmarker] I sh actually should've picked a different one,
[speaker002:] Right?
[speaker005:] because [pause] that could be why the PDA is worse. Because it's further away from most of the people reading digits.
[speaker004:] It's further away. Yeah. Yeah.
[speaker002:] That's probably one of the reasons.
[speaker003:] Hmm. Mm hmm.
[speaker001:] Well, yeah. You could look at, I guess, that PZM or something.
[speaker005:] Yep.
[speaker002:] But the other is, it's very, uh [disfmarker] I mean, even though there's [disfmarker] I'm sure the f f the [disfmarker] the SRI, uh, front end has some kind of pre emphasis, it's [disfmarker] it's, uh [disfmarker] [vocalsound] still, th it's picking up lots of low frequency energy.
[speaker006:] Mm hmm.
[speaker002:] So, even discriminating against it, I'm sure some of it's getting through. Um. But, yeah, you're right. Prob a part of it is just the distance.
[speaker001:] And aren't these pretty bad microphones?
[speaker005:] Yep.
[speaker001:] I mean [disfmarker]
[speaker002:] Well, they're bad. But, I mean, if you listen to it, it sounds OK. You know?
[speaker005:] Yeah. When you listen to it, uh, the PZM and the PDA [disfmarker] Yeah, th the PDA has higher sound floor
[speaker002:] u Yeah.
[speaker005:] but not by a lot. It's really pretty [disfmarker] uh, pretty much the same.
[speaker001:] I just remember you saying you got them to be cheap on purpose. Cheap in terms of their quality. So.
[speaker002:] Well, they're [pause] twenty five cents or so.
[speaker005:] Th we wanted them to be [disfmarker] to be typical of what would be in a PDA.
[speaker002:] Yeah.
[speaker005:] So they are [disfmarker] they're not the PZM three hundred dollar type.
[speaker001:] Mm hmm.
[speaker005:] They're the twenty five cent,
[speaker002:] Yeah.
[speaker005:] buy them in packs of thousand type.
[speaker002:] But, I mean, the thing is people use those little mikes for everything
[speaker001:] I see.
[speaker005:] Everything.
[speaker002:] because they're really not bad. I mean, if you're not [vocalsound] doing something ridiculous like feeding it to a speech recognizer, they [disfmarker] they [disfmarker] [vocalsound] they [disfmarker] you know, you can hear the sou hear the sounds just fine.
[speaker001:] Mm hmm. Right.
[speaker002:] You know, it's [disfmarker] They [disfmarker] I mean, i it's more or less the same principles as these other mikes are built under, it's just that [pause] there's less quality control. They just, you know, churn them out and don't check them. Um. So. So that was [disfmarker] Yeah. So that was i interesting result. So like I said, the front end guys are very much interested in [disfmarker] in this is as [disfmarker] as well and
[speaker006:] So [disfmarker] so, but where is this now? I mean, what's [disfmarker] where do we go from here? I mean,
[speaker005:] Yeah. That was gonna be my question.
[speaker006:] we [disfmarker] so we have a [disfmarker] we have a [disfmarker] a system that works pretty well but it's not, you know, the system that people here are used to using [disfmarker] to working with. So what [disfmarker] what do we do now?
[speaker002:] Well, I think what we wanna do is we want to [disfmarker] eh, and we've talked about this in other [pause] contexts [disfmarker] we want to [vocalsound] have the ability to feed it different features.
[speaker006:] Mm hmm.
[speaker002:] And then, um, [vocalsound] from the point of view of the front end research, it would be s uh, substituting for HTK.
[speaker006:] OK. OK.
[speaker002:] I think that's the key thing. And then if we can feed it different features, then we can try all the different things that we're trying there.
[speaker006:] OK. Alright.
[speaker002:] And then, um, uh, also Dave is [disfmarker] is thinking about using the data in different ways, uh, to [vocalsound] um, uh, explicitly work on reverberation
[speaker006:] Mm hmm.
[speaker002:] starting with some techniques that some other people have [pause] found somewhat useful, and [disfmarker] Yeah.
[speaker006:] OK. So [disfmarker] so the key [pause] thing that's missing here is basically the ability to feed, you know, other features [vocalsound] i into the recognizer and also then to train the system.
[speaker002:] Right. Right.
[speaker006:] OK. And, uh, es I don't know when Chuck will be back but that's exactly what he [disfmarker] he's gonna [disfmarker]
[speaker002:] H h He's [disfmarker] he's sort of back, but he drove for fourteen hours an and wasn't gonna make it in today.
[speaker006:] Oh, OK. So, I think that's one of the things that he said he would be working on. Um. Just sort of t to make sure that [pause] we can do that
[speaker005:] Yeah.
[speaker006:] and [disfmarker] Um.
[speaker002:] Yeah. Right.
[speaker006:] It's [disfmarker] uh, I mean, the [disfmarker] the front end is f i tha that's in the SRI recognizer is very nice in that it does a lot of things on the fly but it unfortunately [pause] is not [pause] designed and, um [disfmarker] [vocalsound] like the, uh, ICSI system is, where you can feed it from a pipeline of [disfmarker] of the command. So, the [disfmarker] what that means probably for the foreseeable future is that you have to, uh, dump out, um [disfmarker] you know, if you want to use some new features, you have to dump them into individual files and [pause] give those files to the recognizer.
[speaker005:] We do [disfmarker] we tend to do that anyway.
[speaker006:] OK.
[speaker005:] Oh. So, although you [disfmarker] you can pipe it as well, we tend to do it that way because that way you can concentrate on one block and not keep re doing it over and over.
[speaker006:] Oh, OK. Alright.
[speaker002:] Yeah. Yeah. So I've [disfmarker] I [disfmarker]
[speaker005:] So tha that's exactly what the P file [pause] is for.
[speaker002:] Yeah.
[speaker006:] Yeah, the [disfmarker] the [disfmarker] the cumbersome thing is [disfmarker] is, um [disfmarker] is that you actually have to dump out little [disfmarker] little files.
[speaker001:] Uh [disfmarker]
[speaker006:] So for each segment that you want to recognize [vocalsound] you have to [pause] dump out [pause] a separate file.
[speaker005:] Uh huh.
[speaker006:] Just like i th like th as if there were these waveform segments, but instead you have sort of feature file segments. But, you know [disfmarker] So.
[speaker002:] Cool. OK. So the s the [disfmarker] the next thing we had on the agenda was something about alignments?
[speaker001:] Oh. Yes, we have [disfmarker] I don't know, did you wanna talk about it, or [disfmarker]? I can give a [disfmarker] I was just telling this to Jane and [disfmarker] and [disfmarker] W we [disfmarker] we were able to get some definite improvement on the forced alignments by looking at them first and then realizing the kinds of errors [pause] that were occurring and um, some of the errors occurring very frequently are just things like the first word being moved to as early as possible in the recognition, which is a um, I think was both a [disfmarker] a pruning [pause] problem and possibly a problem with needing constraints on word locations. And so we tried both of these st things. We tried saying [disfmarker] I don't know, I got this [vocalsound] whacky idea that [disfmarker] just from looking at the data, that when people talk [pause] their words are usually chunked together. It's not that they say one word and then there's a bunch of words together. They're [comment] might say one word and then another word far away if they were doing just backchannels? But in general, if there's, like, five or six words and one word's far away from it, that's probably wrong on average. So, um [disfmarker] And then also, ca the pruning, of course, was too [disfmarker] too severe.
[speaker006:] So that's actually interesting. The pruning was the same value that we used for recognition. And we had lowered that [disfmarker] we had used tighter pruning after Liz ran some experiments showing that, you know, it runs slower and there's no real difference in [disfmarker]
[speaker005:] No gain.
[speaker001:] Actually it was better with [disfmarker] slightly better or about th
[speaker006:] Right.
[speaker001:] it was the same with tighter pruning.
[speaker006:] So for free recognition, this [disfmarker] the lower pruning value is better. You [disfmarker]
[speaker001:] It's probably cuz the recognition's just bad en at a point where it's bad enough that [disfmarker] that you don't lose anything.
[speaker006:] Correct. Right. Um, but it turned out for [disfmarker] for [disfmarker] to get accurate alignments it was really important to open up the pruning significantly.
[speaker001:] Right.
[speaker002:] Hmm.
[speaker006:] Um [pause] because otherwise it would sort of do greedy alignment, um, in regions where there was no real speech yet from the foreground speaker.
[speaker002:] Mm hmm.
[speaker006:] Um, [vocalsound] so that was one big factor that helped improve things and then the other thing was that, you know, as Liz said the [disfmarker] we f enforce the fact that, uh, the foreground speech has to be continuous. It cannot be [disfmarker] you cannot have a background speech hypothesis in the middle of the foreground speech. You can only have background speech at the beginning and the end.
[speaker001:] Yeah. I mean, yeah, it isn't always true, and I think what we really want is some clever way to do this, where, um, you know, from the data or from maybe some hand corrected alignments from transcribers that things like words that do occur just by themselves [pause] a alone, like backchannels or something that we did allow to have background speech around it [disfmarker]
[speaker004:] Yeah.
[speaker001:] those would be able to do that,
[speaker003:] Sorry.
[speaker001:] but the rest would be constrained. So, I think we have a version that's pretty good for the native speakers. I don't know yet about the non native speakers. And, um, we basically also made noise models for the different [disfmarker] sort of grouped some of the [pause] mouth noises together. Um, so, and then there's a background speech model. And we also [disfmarker] There was some neat [disfmarker] or, interesting cases, like there's one meeting where, [vocalsound] um, Jose's giving a presentation and he's talking about, um, the word "mixed [pause] signal" and someone didn't understand, uh, that you were saying "mixed" [disfmarker] I think, Morgan.
[speaker008:] Yeah, yeah.
[speaker001:] And so your speech ch was s saying something about mixed signal. And the next turn was a lot of people saying "mixed", like "he means mixed signal" or "I think it's mixed". And the word "mixed" in this segment occurs, like, a bunch of times.
[speaker008:] Sh
[speaker001:] And Chuck's on the lapel here, and he also says "mixed" but it's at the last one, and of course the aligner th aligns it everywhere else to everybody else's "mixed",
[speaker008:] Yeah.
[speaker001:] cuz there's no adaptation yet. So there's [disfmarker] [vocalsound] I think there's some issues about [disfmarker] u We probably want to adapt at least the foreground speaker. But, I guess Andreas tried adapting both the foreground and a background generic speaker, and that's actually a little bit of a f funky model. Like, it gives you some weird alignments, just because often the background speakers match better to the foreground than the foreground speaker.
[speaker006:] Oh [disfmarker]
[speaker004:] Yeah.
[speaker008:] Oh.
[speaker001:] So there's some things there, especially when you get lots of the same words, uh, occurring in the [disfmarker]
[speaker006:] Well, the [disfmarker] I [disfmarker] I think you can do better by [vocalsound] uh, cloning [disfmarker] so we have a reject phone. And you [disfmarker] and what we wanted to try with [disfmarker] you know, once we have this paper written and have a little more time, [vocalsound] uh, t cloning that reject model and then one copy of it would be adapted to the foreground speaker to capture the rejects in the foreground, like fragments and stuff, and the other copy would be adapted to the background speaker.
[speaker001:] Right. I mean, in general we actually [disfmarker]
[speaker006:] And [disfmarker]
[speaker001:] Right now the words like [pause] partial words are [pause] reject models
[speaker006:] Mm hmm.
[speaker001:] and you normally allow those to match to any word. But then the background speech was also a reject model, and so this constraint of not allowing rejects in between [disfmarker] you know, it needs to differentiate between the two. So just sort of working through a bunch of debugging kinds of issues.
[speaker006:] Right.
[speaker001:] And another one is turns, like people starting with [vocalsound] "well I think" and someone else is [pause] "well how about". So the word "well" is in this [disfmarker] in this [pause] segment multiple times, and as soon as it occurs usually the aligner will try to align it to the first person who says it. But then that constraint of sort of [disfmarker] uh, proximity constraint will push it over to the person who really said it in general.
[speaker005:] Is the proximity constraint a hard constraint, or did you do some sort of probabilistic weighting distance, or [disfmarker]?
[speaker006:] We [disfmarker] we didn't [disfmarker] No.
[speaker001:] Right now it's a kluge.
[speaker006:] We [disfmarker] w OK. We [disfmarker] it's straightforward to actually just have a [disfmarker] a penalty that doesn't completely disallows it but discourages it. But, um, we just didn't have time to play with, you know, tuning yet another [disfmarker] yet another parameter.
[speaker005:] The ve level. Yeah.
[speaker001:] Yeah.
[speaker006:] And really the reason we can't do it is just that we don't have a [disfmarker] we don't have ground truth for these. So, [vocalsound] we would need a hand marked, um, [vocalsound] word level alignments or at least sort of the boundaries of the speech betw you know, between the speakers. Um, and then use that as a reference and tune the parameters of the [disfmarker] of the model, uh, to op to get the best [pause] performance.
[speaker001:] Yeah.
[speaker002:] G given [disfmarker] I [disfmarker] I mean, I wa I wa I was gonna ask you anyway, uh, how you assessed that things were better.
[speaker006:] Mm hmm.
[speaker001:] I looked at them. I spent two days [disfmarker] um, in Waves [disfmarker]
[speaker002:] OK.
[speaker001:] Oh, it was painful because [vocalsound] the thing is, you know the alignments share a lot in common, so [disfmarker] And you're [disfmarker] yo you're looking at these segments where there's a lot of speech. I mean, a lot of them have a lot of words.
[speaker002:] Yeah.
[speaker001:] Not by every speaker but by some speaker there's a lot of words.
[speaker002:] Yeah.
[speaker008:] Ju
[speaker001:] No, not [disfmarker] I mean that if you look at the individual segments from just one person you don't see a lot of words,
[speaker002:] Yeah.
[speaker006:] Mm hmm.
[speaker002:] Yeah.
[speaker001:] but altogether you'll see a lot of words up there. And so the reject is also mapping and pauses [disfmarker]
[speaker004:] Yeah.
[speaker001:] So I looked at them all in Waves and just lined up all the alignments, and, at first it sort of looked like a mess and then the more I looked at it, I thought "OK, well it's moving these words leftward and [disfmarker]" You know, it wasn't that bad. It was just doing certain things wrong. So [disfmarker] But, I don't, you know, have time to l [comment] to look at all of them and it would be really useful to have, like, a [disfmarker] a transcriber who could use Waves, um, just mark, like, the beginning and end of the foreground speaker's real words [disfmarker] like, the beginning of the first word, the end of the last word [disfmarker]
[speaker003:] Yeah.
[speaker001:] and then we could, you know, do some adjustments.
[speaker003:] I [disfmarker] OK. I have to ask you something, is i does it have to be Waves? Because if we could benefit from what you did, incorporate that into the present transcripts, [comment] that would help.
[speaker006:] No.
[speaker003:] And then, um, the other thing is, I believe that I did hand So. One of these transcripts was gone over by a transcriber and then I hand marked it myself so that we do have, uh, the beginning and ending of individual utterances.
[speaker006:] Mm hmm.
[speaker003:] Um, I didn't do it word level, but [disfmarker] but in terms [disfmarker] So I [disfmarker] so for [disfmarker] for one of the N S A groups.
[speaker001:] Mm hmm.
[speaker003:] And also I went back to the original one that I first transcribed and [disfmarker] and did it w uh, w uh, utterance by utterance for that particular one. So I think you do have [disfmarker] if that's a sufficient unit, I think that you do have hand marking for that. But it'd be wonderful to be able to [vocalsound] benefit from your Waves stuff.
[speaker001:] Mm hmm.
[speaker006:] We don't care what [disfmarker] what tool you use.
[speaker001:] Yeah. I mean, if [disfmarker] if you can, um [disfmarker] if you wanna [disfmarker]
[speaker003:] OK. I used it in Transcriber
[speaker006:] U uh [disfmarker]
[speaker003:] and it's [disfmarker] it's in the [disfmarker]
[speaker001:] well, Jane and I were [disfmarker] just in terms of the tool, talking about this. I guess Sue had had some [pause] reactions. You know, interface wise if you're looking at speech, you wanna be able to know really where the words are.
[speaker003:] Yeah, that's right.
[speaker001:] And so, [vocalsound] we can give you some examples of sort of what this output looks like,
[speaker003:] Middle of the word, or [disfmarker]
[speaker001:] um, and see if you can in maybe incorporate it into the Transcriber tool some way, or [disfmarker]
[speaker003:] Well, I th I'm thinking just ch e e incorporating it into the representation.
[speaker001:] Um.
[speaker003:] I mean, if it's [disfmarker] if it's [disfmarker]
[speaker001:] You mean like [disfmarker] Yeah, word start insights.
[speaker003:] if you have start points, if you have, like, time tags,
[speaker001:] Right.
[speaker003:] which is what I assume. Isn't that what [disfmarker] what you [disfmarker]? Well, see, Adam would be [disfmarker]
[speaker006:] Yeah, whatever you use. I mean, we convert it to this format that the, um, NIST scoring tool unders uh, CTM. Conversation Time Marked file.
[speaker001:] Yeah.
[speaker006:] And [disfmarker] and then that's the [disfmarker] that's what the [disfmarker]
[speaker005:] I think Transcriber, uh, outputs CTM.
[speaker003:] If it [disfmarker]? OK.
[speaker005:] I think so.
[speaker003:] So you would know this more than I would.
[speaker001:] Yeah.
[speaker006:] Right.
[speaker001:] So, I mean [disfmarker]
[speaker003:] It seems like she [disfmarker] if she's g if she's moving time marks around, since our representation in Transcriber uses time marks, it seems like there should be some way of [disfmarker] of using that [disfmarker] benefitting from that.
[speaker005:] Right.
[speaker001:] Yeah, it wou the advantage would just be that when you brought up a bin you would be able [disfmarker] if you were zoomed in enough in Transcriber to see all the words,
[speaker002:] Mm hmm.
[speaker001:] you would be able to, like, have the words sort of located in time, if you wanted to do that.
[speaker002:] So [disfmarker] so if we e e even just had a [disfmarker] a [disfmarker] It sounds like w we [disfmarker] [vocalsound] we almost do.
[speaker001:] So.
[speaker002:] Uh, if we [disfmarker] We have two.
[speaker003:] We have two.
[speaker002:] Yeah. Just ha uh, trying out [pause] the alignment [vocalsound] procedure that you have on that
[speaker001:] Mm hmm.
[speaker002:] you could actually get something, um [disfmarker] uh, uh, get an objective measure.
[speaker006:] Mm hmm.
[speaker002:] Uh [disfmarker]
[speaker001:] You mean on [disfmarker] on the hand marked, um [disfmarker] So we [disfmarker] we only r hav I only looked at actually alignments from one meeting that we chose,
[speaker002:] Yeah.
[speaker001:] I think MR four, just randomly, um [disfmarker] And [disfmarker]
[speaker006:] Actually, not randomly.
[speaker001:] Not randomly [disfmarker]
[speaker006:] We knew [disfmarker] we knew that it had these insertion errors from [disfmarker]
[speaker001:] It had sort of [pause] average recognition performance in a bunch of speakers
[speaker006:] Yeah. Yeah.
[speaker001:] and it was a Meeting Recorder meeting. Um. But, yeah, we should try to use what you have. I did re run recognition on your new version of MR one.
[speaker003:] Oh, good.
[speaker001:] I [disfmarker] I mean the [disfmarker] the one with Dan [pause] Ellis in it [vocalsound] and Eric.
[speaker003:] Good! Uh huh. Yeah, exactly. Yeah. Yeah.
[speaker007:] I don't think that was the new version.
[speaker001:] Um [disfmarker] That [disfmarker] Yeah, actually it wasn't the new new,
[speaker003:] OK.
[speaker001:] it was the medium new. But [disfmarker] but we would [disfmarker] we should do the [disfmarker] the latest version.
[speaker003:] OK.
[speaker007:] Yeah. You [disfmarker] did you adjust the [disfmarker] the utterance times, um, for each channel?
[speaker001:] It was the one from last week.
[speaker003:] Yes. Yes, I did. And furthermore, I found that there were a certain number where [disfmarker] [vocalsound] not [disfmarker] not a lot, but several times I actually [vocalsound] moved an utterance from [vocalsound] Adam's channel to Dan's or from Dan's to Adam's. So there was some speaker identif And the reason was because [vocalsound] I transcribed that at a point before [disfmarker] [vocalsound] uh, before we had the multiple audio available f so I couldn't switch between the audio. I [disfmarker] I transcribed it off of the mixed channel entirely, which meant in overlaps, I was at a [disfmarker] at a terrific disadvantage.
[speaker001:] Right. Right.
[speaker003:] In addition it was before the channelized, uh, possibility was there. And finally I did it using the speakers of my, um [disfmarker] [vocalsound] of [disfmarker] you know, off the CPU on my [disfmarker] on my machine cuz I didn't have a headphone. So it [@ @], like, I mean [disfmarker]
[speaker001:] Right.
[speaker003:] Yeah, I [disfmarker] I mean, i in retrospect [vocalsound] it would've been good to ha [vocalsound] have got I should've gotten a headphone. But in any case, um, thi this is [disfmarker] this was transcribed in a [disfmarker] in a, [vocalsound] uh, less optimal way than [disfmarker] than the ones that came after it, and I was able to [disfmarker] you know, an and this meant that there were some speaker identif identifications
[speaker007:] Well, I know there were some speaker labelling problems, um, after interruptions.
[speaker003:] which were changes. Yeah. Fixed that.
[speaker007:] Is that what you're referring to? I mean, cuz there's this one instance when, for example, you're running down the stairs.
[speaker003:] Oh, well [disfmarker]
[speaker007:] I remember this meeting really well.
[speaker004:] Yeah.
[speaker007:] Right.
[speaker001:] Don [disfmarker] Don has had [disfmarker] [vocalsound] He knows [disfmarker] he can just read it like a play.
[speaker007:] It's a [disfmarker] Yeah, I've [disfmarker] I've [disfmarker] I'm very well acquainted with this meeting. Yeah, I can s
[speaker004:] Yeah.
[speaker001:] "And then she said, and then he said."
[speaker007:] Yeah, I know it by heart. So, um, [vocalsound] there's one point when you're running down the stairs.
[speaker003:] Uh oh.
[speaker007:] Right? And, like, there's an interruption. You interrupt somebody, but then there's no line after that. For example, there's no speaker identification after that line.
[speaker003:] Uh huh.
[speaker007:] Is that what you're talking about? Or were there mislabellings as far as, like, the a Adam was [disfmarker]?
[speaker003:] That was fixed, um, before [disfmarker] i i i I think I I think I understood that pretty [disfmarker]
[speaker007:] Yeah. Cuz I thought I let you know about that.
[speaker003:] Thank you for mentioning.
[speaker007:] Yeah.
[speaker003:] Yeah, no, tha that [disfmarker] That I think went away a couple of versions ago,
[speaker007:] OK. But you're actually saying that certain, uh, speakers were mis mis identified.
[speaker003:] but it's good to know. Yeah. So, with [disfmarker] under [disfmarker] um, uh, listening to the mixed channel, there were times when, as surprising as that is, I got Adam's voice confused with Dan's and vice versa [disfmarker]
[speaker007:] OK.
[speaker003:] not for long utterances,
[speaker007:] OK.
[speaker001:] Yeah.
[speaker003:] but jus just a couple of places,
[speaker002:] Mm hmm.
[speaker003:] and embedde embedded in overlaps. The other thing that was w interesting to me was that I picked up a lot of, um, backchannels which were hidden in the mixed signal, which, you know, I mean, you c not [disfmarker] not too surprising.
[speaker001:] Right.
[speaker003:] But the other thing that [disfmarker] I [disfmarker] I hadn't thought about this, but I thou I wanted to raise this when you were [disfmarker] uh, with respect to also a strategy which might help with the alignments potentially, but that's [disfmarker] When I was looking at these backchannels, they were turning up usually [disfmarker] [vocalsound] very often in [disfmarker] w well, I won't say "usually" [disfmarker] but anyway, very often, I picked them up in a channel [vocalsound] w which was the person who had asked a question. S so, like, someone says "an and have you done the so and so?" And then there would be backchannels, but it would be the person who asked the question. Other people weren't really doing much backchannelling. And, you know, sometimes you have the [disfmarker] Yeah, uh huh.
[speaker001:] Well, that's interesting.
[speaker003:] I mean, i it wouldn't be perfect,
[speaker001:] Yeah.
[speaker003:] but [disfmarker] but it does seem more natural to give a backchannel when [disfmarker] when you're somehow involved in the topic,
[speaker001:] No, that's really interesting.
[speaker003:] and the most natural way is for you to have initiated the topic by asking a question.
[speaker002:] Mm hmm.
[speaker006:] Well, I think [disfmarker]
[speaker001:] That's interesting.
[speaker006:] No. I think it's [disfmarker] actually I think what's going on is backchannelling is something that happens in two party conversations.
[speaker003:] Mm hmm.
[speaker006:] And if you ask someone a question, you essentially initiating a little two party conversation.
[speaker003:] Yeah.
[speaker001:] Well, actu Yeah, when we looked at this [disfmarker]
[speaker006:] So then you're [disfmarker] so and then you're expected to backchannel
[speaker003:] Exactly.
[speaker006:] because the person is addressing you directly and not everybody.
[speaker003:] Exactly. Exactly my point.
[speaker006:] Yeah.
[speaker003:] An and so this is the expectation thing that [disfmarker] uh, uh,
[speaker006:] Yeah.
[speaker001:] Mm hmm.
[speaker006:] Right. Right.
[speaker003:] just the dyadic [disfmarker] But in addition, you know, if someone has done this analysis himself and isn't involved in the dyad, but they might also give backchannels to verify what [disfmarker] what the answer is that this [disfmarker] that the [disfmarker] the answerer's given [disfmarker]
[speaker002:] H I tell you, I say [disfmarker] I say "uh huh" a lot,
[speaker001:] Right. It's [disfmarker]
[speaker003:] There you go.
[speaker001:] Well, but it's interesting cuz, uh [disfmarker]
[speaker002:] while people are talking to each other.
[speaker003:] There you go.
[speaker001:] But there are fewer [disfmarker] I think there are fewer "uh huhs".
[speaker003:] Yeah. Yeah.
[speaker001:] I mean, just from [disfmarker] We were looking at word frequency lists to try to find the cases that we would allow to have the reject words in between in doing the alignment. You know the ones we wouldn't constrain to be next to the other words.
[speaker003:] Oh, yeah.
[speaker001:] And "uh huh" is not as frequent as it sort of would be in Switchboard, if you looked at just a word frequency list of one word short utterances. And "yeah" is way up there, but not "uh huh". And so I was thinking thi it's not like [pause] you're being encouraged by everybody else to keep [pause] talking in the meeting. And uh, that's all, I I'll stop there, cuz I I think what you say makes a lot of sense.
[speaker003:] Well, that's right. And that would [disfmarker] Well, an
[speaker001:] But it was sort of [disfmarker]
[speaker003:] And what you say is the [disfmarker] is the re uh, o other side of this, which is that, you know, so th there are lots of channels where you don't have these backchannels, w when a question has been asked and [disfmarker] and these [disfmarker]
[speaker001:] Right. There's just probably less backchannelling in general,
[speaker003:] Mm hmm. So that's good news, really.
[speaker001:] even if you consider every other person altogether one person in the meeting, but we'll find out anyway. We were [disfmarker] I guess the other thing we're [disfmarker] we're [disfmarker] I should say is that we're gonna, um try [disfmarker] compare this type of overlap analysis to Switchboard,
[speaker006:] And
[speaker001:] where [disfmarker] and CallHome, where we have both sides, so that we can try to answer this question of, you know, [vocalsound] is there really more overlap in meetings or is it just because we don't have the other channel in Switchboard
[speaker005:] Mm hmm.
[speaker002:] Mm hmm.
[speaker001:] and we don't know what people are doing. Try to create a paper out of that.
[speaker002:] Yeah. I mean, y y you folks have probably [pause] already told me, but were [disfmarker] were you intending to do a Eurospeech submission, or [disfmarker]?
[speaker001:] Um, you mean the one due tomorrow?
[speaker002:] Yeah.
[speaker001:] Yeah. Well, we're still, like, writing the scripts for doing the research, and we will [disfmarker] Yes, we're gonna try.
[speaker003:] Mm hmm.
[speaker001:] And I was telling Don, do not [pause] take this as an example of how people should work.
[speaker002:] Do as I say,
[speaker007:] That's r
[speaker001:] So, [comment] we will try.
[speaker002:] don't do as I do. Yeah.
[speaker005:] Well [disfmarker]
[speaker001:] It'll probably be a little late,
[speaker005:] It is different.
[speaker001:] but I'm gonna try it.
[speaker005:] In previous years, Eurospeech only had the abstract due by now, not the full paper.
[speaker007:] Right.
[speaker001:] Right.
[speaker005:] And so all our timing was off. I've given up on trying to do digits. I just don't think that what I have so far makes a Eurospeech paper.
[speaker001:] Well, I'm no We may be in the same position, and I figured [vocalsound] we'll try, because that'll at least get us to the point where we have [disfmarker] We have this really nice database format that Andreas and I were working out that [disfmarker] It [disfmarker] it's not very fancy. It's just a ASCII line by line format, but it does give you information [disfmarker]
[speaker006:] It's the [disfmarker] it's the spurt format.
[speaker001:] It [disfmarker] Yeah, we're calling these "spurts" after Chafe. I was trying to find what's a word for [pause] a continuous region with [pause] pauses around it?
[speaker003:] Hmm.
[speaker002:] Yeah. I know that th the Telecom people use [disfmarker] use "spurt" for that.
[speaker003:] Good.
[speaker001:] They do?
[speaker006:] Oh.
[speaker002:] Yes.
[speaker001:] Oh! Oh.
[speaker002:] And that's [disfmarker] I mean, I [disfmarker] I was using that for a while when I was doing the rate of speech stuff,
[speaker001:] I would jus
[speaker002:] because I [disfmarker] because I looked up in some books and I found [disfmarker] OK, I wanna find a spurt [vocalsound] in which [disfmarker]
[speaker001:] Ah, right! It's just, like, defined by the acoustics.
[speaker002:] and [disfmarker] an because [disfmarker] cuz it's another question about how [pause] many pauses they put in between them.
[speaker005:] Horrible.
[speaker001:] Right.
[speaker002:] But how fast do they do [pause] the words within the spurt?
[speaker001:] Right.
[speaker002:] Yeah.
[speaker007:] you know "Burst" also?
[speaker005:] It's gonna [disfmarker]
[speaker001:] Well, that's what we were calling spurt,
[speaker005:] Burst.
[speaker007:] Isn't "burst" is used also?
[speaker001:] so [disfmarker]
[speaker005:] Spurt has the horrible name overloading with other [disfmarker] with hardware at ICSI.
[speaker002:] Here. Just very locally, yeah. But [disfmarker] but that just [disfmarker]
[speaker001:] Well, well, Chafe had this wor I think it was Chafe, or somebody had a [disfmarker] the word "spurt" originally,
[speaker008:] Here [@ @] [disfmarker]
[speaker001:] and so I [disfmarker]
[speaker003:] Actually [disfmarker]
[speaker001:] But tha that's good to know. Was thi it's Chafe?
[speaker006:] So maybe we should talk [disfmarker]
[speaker003:] Well, see, I know S Sue wrote about spurts of development.
[speaker001:] Maybe it was Sue [disfmarker]?
[speaker003:] But, in any case, I think it's a good term,
[speaker001:] Y
[speaker005:] Hmm!
[speaker003:] and, uh [disfmarker]
[speaker001:] So we have spurts and we have spurt ify dot shell and spurt ify
[speaker002:] Yeah.
[speaker003:] And ma maybe [disfmarker] maybe Chafe did.
[speaker006:] Uh.
[speaker002:] Yeah.
[speaker006:] So s
[speaker003:] I know [disfmarker] I know Ch Chafe dealt with [disfmarker]
[speaker001:] And then it's got all [disfmarker] it's a verb now.
[speaker007:] That's cool.
[speaker006:] W uh, w
[speaker003:] Chafe speaks about intonation units.
[speaker001:] Yes. Right.
[speaker003:] But maybe he speaks about spurts as well
[speaker006:] We
[speaker003:] and I just don't know. Yeah, go ahead.
[speaker006:] So what we're doing [disfmarker]
[speaker005:] I've heard "burst" also.
[speaker006:] uh, this [disfmarker] this is just [disfmarker] maybe someone has s some [disfmarker] some ideas about how to do it better,
[speaker007:] Mmm.
[speaker006:] but we [disfmarker] So we're taking these, uh, alignments from the individual channels. We're [disfmarker] from each alignment we're producing, uh, one of these CTM files, which essentially has [disfmarker] it's just a linear sequence of words with the begin times for every word and the duration.
[speaker003:] Great.
[speaker006:] And [disfmarker] and [disfmarker] and of course [disfmarker]
[speaker001:] It looks like a Waves label file almost. Right?
[speaker006:] Right. But it has [disfmarker] one [disfmarker] the first column has the meeting name,
[speaker001:] It's just [disfmarker]
[speaker006:] so it could actually contain several meetings. Um. And the second column is the channel. Third column is the, um, start times of the words and the fourth column is the duration of the words. And then we're, um [disfmarker] OK. Then we have a messy alignment process where we actually insert into the sequence of words the, uh, tags for, like, where [disfmarker] where sentence [disfmarker] ends of sentence, question marks, um, [vocalsound] various other things.
[speaker001:] Yeah. These are things that we had Don [disfmarker]
[speaker006:] Uh.
[speaker001:] So, Don sort of, um, propagated the punctuation from the original transcriber [disfmarker]
[speaker006:] Right.
[speaker001:] so whether it was, like, question mark or period or, [vocalsound] um, you know, comma and things like that, and we kept the [disfmarker] and disfluency dashes [disfmarker] uh, kept those in because we sort of wanna know where those are relative to the spurt overlaps [disfmarker]
[speaker006:] Mm hmm. Right. So [disfmarker] so those are actually sort of retro fitted into the time alignment.
[speaker001:] sp overlaps, or [disfmarker]
[speaker006:] And then we merge all the alignments from the various channels and we sort them by time. And then there's a [disfmarker] then there's a process where you now determine the spurts. That is [disfmarker] Actually, no, you do that before you merge the various channels. So you [disfmarker] you id identify by some criterion, which is pause length [disfmarker] you identify the beginnings and ends of these spurts, and you put another set of tags in there to keep those straight.
[speaker002:] Mm hmm.
[speaker006:] And then you merge everything in terms of, you know, linearizing the sequence based on the time marks. And then [vocalsound] you extract the individual channels again, but this time you know where the other people start and end talking [disfmarker] you know, where their spurts start and end. And so you extract the individual channels, uh, one sp spurt by spurt as it were. Um, and inside the words or between the words you now have begin and end [pause] tags for overlaps. So, you [disfmarker] you basically have everything sort of lined up and in a form where you can look at the individual speakers and how their speech relates to the other speakers' speech.
[speaker005:] Right.
[speaker001:] Uh, I mean, I think that's actually really u useful also
[speaker006:] And [disfmarker]
[speaker001:] because even if you weren't studying overlaps, if you wanna get a transcription for the far field mikes, how are you gonna know which words from which speakers occurred at which times relative to each other? You have to be able to [pause] get a transcript like [disfmarker] like this anyway, just for doing far field recognition. So, you know, it's [disfmarker] it's sort of [disfmarker]
[speaker006:] Yeah.
[speaker001:] I thi it's just an issue we haven't dealt with before, how you time align things that are overlapping anyway.
[speaker006:] So [disfmarker]
[speaker003:] That's wonderful.
[speaker005:] Well [disfmarker]
[speaker006:] And [disfmarker] and we [disfmarker]
[speaker001:] I mean, i I never thought about it before,
[speaker005:] Y yes.
[speaker006:] In [disfmarker]
[speaker001:] but [disfmarker]
[speaker005:] I mean, s when I came up with the original data [disfmarker] suggested data format based on the transcription graph, there's capability of doing that sort of thing in there.
[speaker001:] Right.
[speaker003:] Mm hmm.
[speaker001:] But you can't get it directly from the transcription.
[speaker006:] Right.
[speaker003:] Yeah, that's right.
[speaker006:] Well, this is [disfmarker] this is just [disfmarker]
[speaker001:] Yeah, this is like a poor man's ver formatting version. But it's, you know [disfmarker] It's clean, it's just not fancy.
[speaker005:] Right.
[speaker006:] Well, there's lots of little things.
[speaker001:] Um.
[speaker006:] It's like there're twelve different scripts which you run and then at the end you have what you want. But, um, at the very last stage we throw away the actual time information. All we care about is whether [disfmarker] that there's a certain word was overlapped by someone else's word. So you sort of [disfmarker] at that point, you discretize things into just having overlap or no overlap. Because we figure that's about the level of analysis that we want to do for this paper.
[speaker005:] Mm hmm.
[speaker006:] But if you wanted to do a more fine grained analysis and say, you know, how far into the word is the overlap, you could do that. It's just [disfmarker] it'll just require more [disfmarker]
[speaker001:] Yeah. Just [pause] sort of huge.
[speaker006:] you know, slightly different [disfmarker]
[speaker003:] What's interesting is it's exactly what, um, i in discussing with, um, Sue about this,
[speaker001:] Yeah.
[speaker003:] um, she, um, i i i indicated that that [disfmarker] you know, that's very important for overlap analysis.
[speaker001:] Yeah. It's [disfmarker] it's nice to know,
[speaker006:] Right.
[speaker001:] and also I think as a human, like, I don't always hear these in the actual order that they occur. So I can have two foreground speakers, you know, Morgan an and [vocalsound] um, Adam and Jane could all be talking, and I could align each of them to be starting their utterance at the correct time, and then look where they are relative to each other, and that's not really what I heard.
[speaker003:] And that's another thing she said.
[speaker001:] Cuz it's just hard to do.
[speaker003:] This is [disfmarker] This is Bever's [disfmarker] Bever's effect, when [disfmarker] where [disfmarker] In psy ps psycho linguistics you have these experiments where people have perceptual biases a as to what they hear,
[speaker001:] Y Yeah. It's sort of [disfmarker]
[speaker003:] that [disfmarker] that [disfmarker] Not the best [disfmarker]
[speaker001:] Yeah, you sort of move things around until you get to a [pause] low information point and yo then you can bring in the other person. So it's [vocalsound] actually not even possible, I think, for any person to listen to a mixed signal, even equalize, and make sure that they have all the words in the right order.
[speaker003:] Mm hmm.
[speaker001:] So, I guess, we'll try to write this Eurospeech paper.
[speaker003:] Superb.
[speaker001:] I mean, we will write it. Whether they accept it [pause] late or not, I don't know. Um, and the good thing is that we have [disfmarker] It's sort of a beginning of what Don can use to link the prosodic features from each file to each other.
[speaker006:] Yeah.
[speaker002:] Yeah. That's the good thing about these pape
[speaker006:] Plus, mayb
[speaker001:] So.
[speaker008:] Hmm?
[speaker001:] i You know, might as well. We I ju Otherwise we won't get the work done [comment] [vocalsound] on our deadline.
[speaker006:] I don't know, m I mean, u u Jane likes to look at data.
[speaker002:] Yeah.
[speaker006:] Maybe, you know, you could [disfmarker] you could look at this format and see if you find anything interesting. I don't know.
[speaker002:] Yeah.
[speaker003:] Well, what I'm thinking is [disfmarker]
[speaker002:] No, it's [disfmarker] that's the good thing about these pape paper deadlines and, uh, you know, class projects, and [disfmarker] [vocalsound] and things like that,
[speaker001:] Yeah.
[speaker006:] Yeah.
[speaker003:] Yeah.
[speaker006:] Mm hmm.
[speaker001:] Right.
[speaker003:] Well, my [disfmarker]
[speaker006:] Well th th the other thing that [disfmarker] that [disfmarker] that yo that you usually don't tell your graduate students is that these deadlines are actually not that, um, you know, strictly enforced,
[speaker002:] because you [disfmarker] you really get g
[speaker003:] Yeah.
[speaker001:] Forces you to do the work.
[speaker002:] Yeah.
[speaker001:] Exactly.
[speaker005:] Strict.
[speaker006:] because [pause] the [disfmarker]
[speaker002:] Oh, now it's out in the public, this [disfmarker] this [disfmarker] this secret information.
[speaker006:] because [disfmarker]
[speaker003:] I think we can ha
[speaker002:] Yeah.
[speaker006:] bec b [vocalsound] Nah [disfmarker]
[speaker001:] Right.
[speaker005:] No.
[speaker001:] So [disfmarker]
[speaker002:] No.
[speaker003:] Nah.
[speaker006:] i Because these [disfmarker] the conference organizers actually have an interest in getting lots of submissions.
[speaker001:] Right.
[speaker006:] I mean, a [disfmarker] a monetary interest.
[speaker005:] Right.
[speaker002:] Yeah.
[speaker006:] So [disfmarker] [vocalsound] Um.
[speaker002:] Th that's [disfmarker] that's true.
[speaker003:] And good ones, good ones,
[speaker006:] And good submission
[speaker003:] which sometimes means [pause] a little extra time.
[speaker002:] That's [disfmarker]
[speaker006:] Right. Well [disfmarker]
[speaker002:] That's true.
[speaker006:] That's another issue, but [disfmarker]
[speaker002:] By th by the way, this is totally unfair, you may [disfmarker] you may feel, but the [disfmarker] the, uh [disfmarker] the morning meeting folks actually have an [disfmarker] an extra month or so.
[speaker006:] Mm hmm.
[speaker004:] Yep.
[speaker005:] Yep. The Aurora [disfmarker] there's a special Aurora [disfmarker]
[speaker006:] When [disfmarker]
[speaker001:] Uh [disfmarker]
[speaker002:] There's a special Aurora session and the Aurora pe people involved in Aurora have till Ma uh, early May [pause] or something to turn in their paper.
[speaker001:] Oh.
[speaker006:] Mmm.
[speaker001:] Oh.
[speaker006:] Mmm. Well, then you can just [disfmarker]
[speaker001:] Oh, well maybe we'll submit to s [comment] [vocalsound] Actually [disfmarker]
[speaker006:] Maybe you can submit the digits paper on e for the Aurora session.
[speaker008:] Yeah.
[speaker004:] Yeah.
[speaker001:] Yeah.
[speaker005:] Oh, I could!
[speaker002:] I if it w
[speaker001:] Yeah.
[speaker005:] I could submit that to Aurora.
[speaker006:] Yeah.
[speaker002:] Well [disfmarker]
[speaker005:] That would be pretty [disfmarker] pretty [disfmarker]
[speaker002:] i it has [disfmarker]
[speaker001:] Yeah.
[speaker005:] S That wouldn't work.
[speaker002:] No, it wouldn't work.
[speaker005:] It's not Aurora.
[speaker002:] It's [disfmarker] it's not the Aurora [disfmarker] I mean, it [disfmarker] it's [disfmarker] it's actually the Aurora task.
[speaker001:] Maybe they'll get s
[speaker005:] Aurora's very specific.
[speaker001:] Well, maybe it won't be after this [vocalsound] deadline [pause] extension.
[speaker002:] It
[speaker006:] But [disfmarker] but the people [disfmarker] I mean, a [disfmarker] a paper that is not on Aurora would probably be more interesting at that point
[speaker001:] Maybe they'll [disfmarker]
[speaker006:] because everybody's so sick and tired of the Aurora task.
[speaker004:] Yeah.
[speaker005:] Oh, I thought you meant this was just the digits section. I didn't know you meant it was Aurora digits.
[speaker002:] Yeah.
[speaker006:] Well, no. If you [disfmarker] if you have [disfmarker] it's to [disfmarker] if you discuss some relation to the Aurora task, like if you use the same [disfmarker]
[speaker002:] This is not the Aurora task. So they just do a little grep for [disfmarker]
[speaker001:] Do [disfmarker] uh, d d
[speaker006:] Um.
[speaker001:] Do not [disfmarker] do not [disfmarker] we are not setting a good example.
[speaker006:] Well, a relation other than negation, maybe,
[speaker001:] This is not a [disfmarker]
[speaker006:] um. So. I don't know.
[speaker001:] Anyway. But the good thing is this does [disfmarker]
[speaker005:] Well, I I don't know. I mean, you could [disfmarker] you could do a paper on [pause] what's wrong with the Aurora task by comparing it to [pause] other ways of doing it.
[speaker006:] How well does an Aurora system do on [disfmarker] on [disfmarker] you know, on digits collected in a [disfmarker] in this environment?
[speaker005:] Different way. Yeah.
[speaker006:] Yeah.
[speaker002:] Maybe.
[speaker006:] Maybe.
[speaker005:] Pretty hokey.
[speaker002:] I think it's a littl little far fetched. Nah, I mean, the thing is Aurora's pretty closed community.
[speaker005:] Yep.
[speaker002:] I mean, you know, the people who were involved in the [disfmarker] [vocalsound] the only people who are allowed to test on that are people who [disfmarker] who made it above a certain threshold in the first round,
[speaker006:] Mm hmm.
[speaker005:] It's very specific.
[speaker002:] uh [vocalsound] w in ninety nine and it's [disfmarker] it's sort of a [disfmarker] it's [disfmarker]
[speaker006:] Well, that's maybe why they don't f know that they have a crummy system.
[speaker002:] not like a [disfmarker]
[speaker006:] I mean, a crummy back end. No, I mean [disfmarker] I mean, seriously, if you [disfmarker] if you have a very [disfmarker] No, I'm sorry.
[speaker001:] Uh, [comment] "beep" [vocalsound] "bee"
[speaker006:] No. I didn't mean anybody [disfmarker] any particular system.
[speaker005:] I mean, th
[speaker006:] I meant this H T K back end. If they [disfmarker]
[speaker002:] Oh, you don't like HTK?
[speaker008:] Yeah.
[speaker006:] I don't h I don't have any stock in HTK or Entropic or anything.
[speaker002:] No. I mean, this [disfmarker] it it's the HTK [pause] that is trained on a very limited amount of data.
[speaker005:] It's d it's very specific.
[speaker006:] Right.
[speaker002:] Yeah.
[speaker006:] But so, if you [disfmarker] But maybe you should, you know, consider more [disfmarker] using more data, or [disfmarker] I mean [disfmarker]
[speaker002:] Oh, yeah. I [disfmarker] I really think that that's true.
[speaker006:] If yo if you sort of hermetically stay within one task and don't look left and right, then you're gonna [disfmarker]
[speaker002:] And they i i
[speaker005:] But they [disfmarker] they had [disfmarker]
[speaker002:] i But [disfmarker]
[speaker005:] They had something very specific in mind when they designed it. Right?
[speaker006:] Right.
[speaker002:] Well, u i
[speaker005:] And so [disfmarker] so you can [disfmarker] you can argue about maybe that wasn't the right thing to do, but, you know, they [disfmarker] they [disfmarker] they had something specific.
[speaker002:] But, one of the reasons I have Chuck's messing around with [disfmarker] with the back end that you're not supposed to touch [disfmarker]
[speaker006:] Mm hmm.
[speaker002:] I mean, for the evaluations, yes, we'll run a version that hasn't been touched.
[speaker006:] Mm hmm.
[speaker002:] But, uh, one of the reasons I have him messing around with that, because I think it's sort of an open question that we don't know the answer to. People always say very glibly [vocalsound] that i if you s show improvement on a bad system, that doesn't mean anything, cuz it may not be [disfmarker] [vocalsound] show [disfmarker] uh, because, you know, it doesn't tell you anything about the good system.
[speaker006:] Mm hmm.
[speaker002:] And I [disfmarker] I've always sort of felt that that depends. You know, that if some peopl If you're actually are getting at something that has some [pause] conceptual substance to it, it will port.
[speaker006:] Mm hmm.
[speaker002:] And in fact, most methods that people now use were originally tried with something that was not their absolute [pause] best system at some level. But of course, sometimes it doesn't, uh, port. So I think that's [disfmarker] that's an interesting question. If we're getting [pause] three percent error on, uh, u uh, English, uh, nati native speakers, [vocalsound] um, using the Aurora system, and we do some improvements and bring it from three to two, [vocalsound] do those same improvements bring, uh, th you know, the SRI system from one point three to [disfmarker] you know, to [vocalsound] point eight?
[speaker006:] Hmm. Mm hmm.
[speaker005:] Zero.
[speaker002:] Well.
[speaker006:] Mmm.
[speaker002:] You know, so that's [disfmarker] that's something we can test.
[speaker006:] Right.
[speaker002:] So. Anyway.
[speaker006:] OK.
[speaker002:] I think we've [disfmarker] [vocalsound] we've covered that one up extremely well.
[speaker003:] Mm hmm.
[speaker006:] Whew!
[speaker002:] OK. So, um [disfmarker] Yeah. So tha so we'll [disfmarker] you know, maybe you guys'll have [disfmarker] have one. Uh, you [disfmarker] you and, uh [disfmarker] and Dan have [disfmarker] have a paper that [disfmarker] that's going in.
[speaker004:] Yeah. Yeah.
[speaker002:] You know, that's [disfmarker] that's pretty solid, on the segmentation [pause] stuff.
[speaker004:] Yeah. I will send you the [disfmarker] the final version,
[speaker002:] Yeah. And the Aurora folks here will [disfmarker] will definitely get something in on Aurora,
[speaker004:] which is not [disfmarker]
[speaker006:] Actually this [disfmarker] this, um [disfmarker] So, there's another paper.
[speaker002:] so.
[speaker006:] It's a Eurospeech paper but not related to meetings. But it's on digits. So, um, uh, a colleague at SRI developed a improved version of MMIE training.
[speaker002:] Uh huh.
[speaker006:] And he tested it mostly on digits because it's sort of a [disfmarker] you know, it doesn't take weeks to train it.
[speaker002:] Right.
[speaker006:] Um. And got some very impressive results, um, with, you know, discriminative, uh, Gaussian training. Um, you know, like, um, error rates [pause] go from [disfmarker] I don't know, in very noisy environment, like from, uh, uh [disfmarker] I for now I [disfmarker] OK, now I have the order of magnit I'm not sure about the order of magnitude. Was it like from ten percent to [vocalsound] eight percent or from e e you know, point [disfmarker] you know, from one percent to point eight percent?
[speaker002:] H i it got [disfmarker] it got better.
[speaker006:] I mean, it's a [disfmarker] It got better.
[speaker004:] Yeah.
[speaker006:] That's the important thing.
[speaker002:] Yeah, yeah.
[speaker006:] Yeah.
[speaker005:] Hey, that's the same percent relative,
[speaker006:] But it's [disfmarker]
[speaker005:] so [disfmarker]
[speaker006:] Yeah. Right.
[speaker002:] Yeah.
[speaker006:] It's, uh, something in [disfmarker]
[speaker002:] Yeah.
[speaker006:] Right.
[speaker005:] Twenty percent relative gain.
[speaker006:] Yeah.
[speaker002:] Yeah. Yeah. Um, [vocalsound] let's see. I think the only thing we had left was [disfmarker] unless somebody else [disfmarker] Well, there's a couple things. Uh, one is [pause] anything that, um, [vocalsound] anybody has to say about Saturday? Anything we should do in prep for Saturday? Um [disfmarker] I guess everybody knows about [disfmarker] I mean, u um, Mari was asking [disfmarker] was trying to come up with something like an agenda and we're sort of fitting around people's times a bit. But, um, [vocalsound] clearly when we actually get here we'll [pause] move things around this, as we need to, but [disfmarker]
[speaker004:] OK.
[speaker002:] so you can't absolutely count on it.
[speaker004:] Yeah.
[speaker002:] But [disfmarker] but, uh [disfmarker]
[speaker001:] Are we meeting in here probably or [disfmarker]?
[speaker002:] Yeah.
[speaker001:] OK.
[speaker002:] That was my thought. I think this is [disfmarker]
[speaker006:] Are we recording it?
[speaker001:] Yeah. We won't have enough microphones,
[speaker002:] u No. I [disfmarker] I hadn't in intended to.
[speaker001:] but [disfmarker] There's no way.
[speaker006:] OK.
[speaker002:] We won we wanna [disfmarker] I mean, they're [disfmarker] there's gonna be, uh, Jeff, Katrin, Mari and two students. So there's five [pause] from there.
[speaker005:] And Brian.
[speaker006:] But you know th
[speaker002:] And Brian's coming, so that's six.
[speaker006:] Mm hmm.
[speaker005:] And plus all of us.
[speaker006:] Can use the Oprah mike.
[speaker002:] Uh [disfmarker]
[speaker001:] Depends how fast you can [pause] throw it.
[speaker005:] It seems like too many [disfmarker] too much coming and going.
[speaker001:] It's just [disfmarker]
[speaker006:] Mm hmm.
[speaker001:] Yeah.
[speaker002:] Well [disfmarker]
[speaker001:] We don't even have enough channel [disfmarker]
[speaker006:] Because it would be a different kind of meeting,
[speaker004:] Yeah.
[speaker006:] that's what I'm [disfmarker]
[speaker008:] Yeah.
[speaker002:] Well [disfmarker]
[speaker006:] But [disfmarker]
[speaker002:] I hadn't [pause] really thought of it,
[speaker006:] Maybe just [disfmarker] maybe not the whole day
[speaker002:] but [disfmarker]
[speaker006:] but just, you know, maybe some [disfmarker] I mean,
[speaker002:] Maybe part of it.
[speaker006:] part of it?
[speaker002:] Maybe part of it.
[speaker005:] Make everyone read digits.
[speaker002:] At the same time.
[speaker005:] At the same time.
[speaker006:] Please.
[speaker001:] At the same time.
[speaker002:] Yeah.
[speaker001:] We c
[speaker002:] I don't know. Any
[speaker001:] That's their initiation into our
[speaker005:] Into our [disfmarker] our [disfmarker] our cult.
[speaker001:] w Yeah, our [disfmarker] Yeah, our [disfmarker]
[speaker006:] Maybe the sections that are not right afte you know, after lunch when everybody's still munching and [disfmarker]
[speaker002:] OK.
[speaker001:] So can you send out a schedule once you know it, jus?
[speaker002:] Well [disfmarker] OK. Yeah. I guess I sent it around a little bit.
[speaker001:] Is [disfmarker] is there a r? There's a res
[speaker002:] But [disfmarker]
[speaker001:] Is it changed now, or [disfmarker]?
[speaker002:] I hadn't heard back from Mari after I [disfmarker] I u u uh, brought up the point abou about Andreas's schedule. So, [vocalsound] um, maybe when I get back there'll be [pause] some [disfmarker] some mail from her.
[speaker001:] OK.
[speaker002:] So, I'll make a [disfmarker]
[speaker003:] I'm looking forward to seeing your representation. That'd be, uh [disfmarker]
[speaker001:] And w we should get [pause] the two meetings from y
[speaker003:] I'd like to see that. Yeah.
[speaker001:] I mean, I know about the first meeting, um, but the other one that you did, the NSA one, which we [pause] hadn't done cuz we weren't running recognition on it,
[speaker003:] Mm hmm.
[speaker001:] because the non native speaker [disfmarker] there were five non native speakers.
[speaker003:] Mm hmm. I see. Mm hmm.
[speaker001:] But, it would be useful for the [disfmarker] to see what we get [pause] with that one.
[speaker003:] Great.
[speaker001:] So.
[speaker003:] OK. It's, uh, two thousand eleven twenty one one thousand.
[speaker001:] Yeah, three. Right. So [disfmarker]
[speaker003:] Great. I sent email when I finished the [disfmarker] that one.
[speaker001:] N S A three, I think.
[speaker003:] That was sort of son Yeah, that's right. That's right. That's much simpler.
[speaker001:] I don't know what they said but I know the number.
[speaker002:] Th that part's definitely gonna confuse somebody who looks at these later.
[speaker006:] Right. Um.
[speaker002:] I mean, this is [disfmarker] we we're recording secret NSA meetings?
[speaker006:] Not the [disfmarker]
[speaker002:] I mean, it's [disfmarker]
[speaker006:] Yeah. Uh.
[speaker003:] Yeah. Not that NSA.
[speaker006:] The [disfmarker] th the [disfmarker]
[speaker002:] It's network services and applications.
[speaker001:] They are hard to understand.
[speaker006:] Wait.
[speaker001:] They're very, uh, out there.
[speaker006:] The [disfmarker]
[speaker001:] I have no idea what they're talking about.
[speaker006:] The, um [disfmarker]
[speaker002:] Yeah.
[speaker006:] th the other good thing about the alignments is that, um, it's not always the machine's fault if it doesn't work. So, you can actually find, um,
[speaker001:] It's the person's fault.
[speaker006:] problem [disfmarker] uh, proble
[speaker001:] It's Morgan's fault.
[speaker006:] You can find [disfmarker] You can find, uh, problems with [disfmarker] with the transcripts,
[speaker002:] It's always Morgan's fault.
[speaker006:] um, you know,
[speaker005:] Oh.
[speaker006:] and go back and fix them.
[speaker001:] Yeah.
[speaker006:] But [disfmarker]
[speaker001:] Tha There are some cases like where the [disfmarker] the wrong speaker [disfmarker] uh, these ca Not a lot, but where the [disfmarker] the wrong person [disfmarker] the [disfmarker] the speech is addre attached to the wrong speaker and you can tell that when you run it. Or at least you can get [pause] clues to it.
[speaker003:] Interesting. I guess it does w
[speaker001:] So these are from the early transcriptions that people did on the mixed signals, like what you have.
[speaker003:] Mm hmm. It also raises the possibility of, um, using that kind of representation [disfmarker] I mean, I don't know, this'd be something we'd wanna check, [comment] but maybe using that representation for data entry and then displaying it on the channelized, uh, representation, cuz it [disfmarker] I think that the [disfmarker] I mean, my [disfmarker] my preference in terms of, like, looking at the data is to see it [pause] in this kind of musical score format.
[speaker001:] Mm hmm.
[speaker003:] And also, s you know, Sue's preference as well.
[speaker001:] Yeah, if you can get it to [disfmarker]
[speaker003:] And [disfmarker] and [disfmarker] but, I mean, this [disfmarker] if this is a better interface for making these kinds of, uh, you know, lo clos local changes, then that'd be fine, too. I don't [disfmarker] I have no idea. I think this is something that would need to be checked. Yeah.
[speaker002:] OK. Th the other thing I had actually was, I [disfmarker] I didn't realize this till today, but, uh, this is, uh, Jose's last day.
[speaker008:] Is my last [disfmarker] my last day.
[speaker005:] Yeah.
[speaker001:] Oh!
[speaker006:] Oh!
[speaker003:] Oh.
[speaker008:] My [disfmarker] [vocalsound] my last meeting [pause] about meetings.
[speaker005:] You're not gonna be here tomorrow? Oh, that's right.
[speaker008:] Yeah.
[speaker005:] Tomorrow [disfmarker]
[speaker008:] Because, eh, I leave, eh, the next Sunday.
[speaker004:] The last meeting meeting?
[speaker005:] It's off.
[speaker008:] I will come back to home [disfmarker] to Spain.
[speaker006:] Mm hmm.
[speaker001:] Oh.
[speaker002:] Yeah. I d so I [disfmarker] I jus
[speaker006:] Mm hmm.
[speaker008:] And I [disfmarker] I would like to [disfmarker] to [disfmarker] to say thank you very much, eh, to all people [pause] in the group and at ICSI,
[speaker001:] Oh.
[speaker006:] Mm hmm.
[speaker005:] Yeah. It was good having you.
[speaker006:] Mmm.
[speaker001:] Yeah.
[speaker008:] because I [disfmarker] I enjoyed [@ @] very much,
[speaker006:] Mmm.
[speaker008:] uh. And I'm sorry by the result of overlapping, because, eh, [vocalsound] I haven't good results, eh, yet but, eh, [vocalsound] I [disfmarker] [vocalsound] I pretend [comment] to [disfmarker] to continuing out to Spain, eh, during the [disfmarker] the following months, eh, because I have, eh, another ideas
[speaker002:] Uh huh.
[speaker008:] but, eh, I haven't enough time to [disfmarker] to [disfmarker] [vocalsound] with six months it's not enough to [disfmarker] [vocalsound] to [disfmarker] to research,
[speaker005:] Yep.
[speaker002:] Yeah.
[speaker008:] eh, and e i I mean, if, eh, the topic is, eh, so difficult, uh, in my opinion, there isn't [disfmarker]
[speaker002:] Yeah. Maybe somebody else will come along and will be, uh, interested in working on it and could start off from where you are also, you know. They'd make use of [disfmarker] of what you've done.
[speaker008:] Yeah. Yeah.
[speaker002:] Yeah.
[speaker008:] But, eh, I [disfmarker] I will try to recommend, eh, at, eh, [vocalsound] the Spanish government but, eh, the following [@ @] scholarship, eh, eh, [vocalsound] eh, will be here [pause] more time, because eh, i in my opinion is [disfmarker] is better, [vocalsound] eh, for us [pause] to [disfmarker] to spend more time here and to work more time i i in a topic. No?
[speaker002:] Yeah, it's a very short time.
[speaker008:] But, uh [disfmarker]
[speaker002:] Yeah.
[speaker005:] Yeah, six months is hard.
[speaker002:] Yeah.
[speaker008:] Yeah. It is.
[speaker005:] I think a year is a lot better.
[speaker008:] Yeah.
[speaker002:] Yeah.
[speaker008:] It's difficult. You [disfmarker] e you have, eh [disfmarker] you are lucky, and you [disfmarker] you find a solution [comment] in [disfmarker] in [disfmarker] in some few tim uh, months, eh? OK. But, eh, I think it's not, eh, common. But, eh, anyway, thank you. Thank you very much. Eh, I [disfmarker] I bring the chocolate, eh, to [disfmarker] [vocalsound] [vocalsound] to tear, uh, with [disfmarker] with you,
[speaker006:] Mmm.
[speaker003:] Ah.
[speaker001:] Oh.
[speaker003:] Nice.
[speaker008:] uh. I [disfmarker] I hope if you need, eh, something, eh, from us in the future, I [disfmarker] I will be at Spain, [vocalsound] to you help, uh.
[speaker005:] Great.
[speaker002:] Well.
[speaker003:] Great.
[speaker002:] Thank you, Jose.
[speaker003:] Thank you.
[speaker001:] Right.
[speaker008:] And, thank you very much.
[speaker006:] Have a good trip.
[speaker002:] Yeah.
[speaker008:] Thank you.
[speaker003:] Yeah.
[speaker006:] Keep in touch.
[speaker002:] Yeah. OK. I guess, uh, unless somebody has something else, we'll read [disfmarker] read our digits
[speaker005:] Digits?
[speaker002:] and we'll get our [disfmarker]
[speaker004:] Uh.
[speaker005:] Are we gonna do them simultaneously
[speaker004:] Oops.
[speaker002:] get our last bit of, uh, Jose's [disfmarker] Jose [disfmarker] Jose's digit [disfmarker]
[speaker005:] or [disfmarker]?
[speaker008:] You [disfmarker] eh [disfmarker]
[speaker002:] Uh, I'm sorry?
[speaker008:] Ye ye you prefer, eh, to eat, eh, chocolate, eh, at the coffee break, eh, at the [disfmarker]? [vocalsound] Or you prefer now, before after [disfmarker]?
[speaker003:] Well, we have a time [disfmarker]
[speaker006:] No, we prefer to keep it for ourselves.
[speaker004:] During [disfmarker]
[speaker006:] Yeah, yeah.
[speaker003:] Well, we have a s a time [disfmarker] time constraint.
[speaker004:] during digits.
[speaker002:] So keep it away from that end of the table.
[speaker003:] Yeah.
[speaker006:] Yeah.
[speaker008:] Yeah.
[speaker003:] Yeah.
[speaker001:] Why is it that I can read your mind?
[speaker005:] Well, we've gotta wait until after di after we take the mikes off.
[speaker004:] No, no.
[speaker005:] So are we gonna do digits simultaneously
[speaker002:] Well?
[speaker001:] You [disfmarker] This is our reward if we [pause] do our digi
[speaker004:] Yeah.
[speaker005:] or what?
[speaker002:] Yeah.
[speaker003:] OK.
[speaker004:] Simultaneous digit chocolate task.
[speaker008:] I [disfmarker] I think, eh, it's enough, eh, for more peopl for more people [pause] after.
[speaker002:] We're gonna [disfmarker] we're gonna do digits at the same [disfmarker]
[speaker001:] Oh.
[speaker006:] Mmm!
[speaker008:] But, eh [disfmarker]
[speaker003:] That's nice.
[speaker006:] Mm hmm.
[speaker002:] Um.
[speaker001:] Oh, thanks, Jose.
[speaker003:] Wow.
[speaker008:] To Andreas, the idea is [disfmarker] is good. [vocalsound] s To eat here.
[speaker002:] Well [disfmarker]
[speaker006:] Mmm.
[speaker003:] Wow.
[speaker006:] Oh.
[speaker003:] Very nice.
[speaker002:] Tha that's [disfmarker] that looks great.
[speaker001:] Oh, wow.
[speaker006:] Oh, yeah. Th it doesn't [disfmarker] it won't leave this room.
[speaker002:] Alright, so in the interest of getting to the [disfmarker]
[speaker001:] We could do digits while other people eat.
[speaker004:] Yeah. Yeah.
[speaker001:] So it's background crunching.
[speaker008:] Yeah.
[speaker006:] Mmm.
[speaker008:] Is, eh, a [disfmarker] another acoustic event.
[speaker003:] Nice.
[speaker001:] We don't have background chewing.
[speaker004:] Background crunch. Yeah.
[speaker001:] No, we don't have any data with background eating.
[speaker006:] Mmm.
[speaker004:] Yeah.
[speaker001:] I'm serious. You
[speaker002:] She's [disfmarker] she's serious.
[speaker005:] It's just the rest of the digits [disfmarker] the rest of the digits are very clean,
[speaker001:] I am serious.
[speaker002:] She is serious.
[speaker006:] Mmm.
[speaker008:] Are you [disfmarker]? Oh, they're clean.
[speaker001:] Well [disfmarker]?
[speaker005:] um, without a lot of background noise,
[speaker004:] Yeah!
[speaker001:] And it [disfmarker]
[speaker005:] so I'm just not sure [disfmarker]
[speaker001:] You have to write down, like, while y what you're [disfmarker] what ch chocolate you're eating cuz they might make different sounds, like n nuts [disfmarker] chocolate with nuts, chocolate without nuts.
[speaker003:] Oh.
[speaker004:] Crunchy frogs.
[speaker002:] Um [disfmarker]
[speaker006:] Chocolate adaptation.
[speaker002:] Actually [disfmarker] [vocalsound] actually kind of careful cuz I have a strong allergy to nuts, so I have to sort of figure out one without th
[speaker001:] That w Oh, yeah, they [disfmarker] they might.
[speaker002:] It's hard to [disfmarker] hard to say.
[speaker001:] Maybe those? They're so [disfmarker]
[speaker002:] I don't know.
[speaker001:] I don't know.
[speaker002:] Um [disfmarker]
[speaker001:] This is [disfmarker] You know, this is a different kind of speech,
[speaker002:] Well [disfmarker]
[speaker008:] Take [disfmarker] take several.
[speaker001:] looking at chocolates, deciding [disfmarker]
[speaker006:] Mmm.
[speaker001:] you know, it's another style.
[speaker006:] Mmm.
[speaker002:] Yeah. I may [disfmarker] I may hold off. But if I was [disfmarker] eh, but maybe I'll get some later.
[speaker006:] Mmm.
[speaker002:] Thanks. Well [disfmarker] well, why don't we [disfmarker]? He [disfmarker] he's worried about a ticket. Why don't we do a simultaneous one?
[speaker001:] OK.
[speaker005:] OK.
[speaker003:] OK.
[speaker006:] Mmm.
[speaker002:] Simultaneous one? OK.
[speaker005:] Remember to read the transcript number, please.
[speaker001:] And you laughed at me, too, f the first time I said that.
[speaker006:] Right.
[speaker008:] OK.
[speaker004:] Oops.
[speaker002:] I have to what?
[speaker008:] Yeah.
[speaker001:] You laughed at me, too, the first time I sa said [disfmarker]
[speaker002:] I did, and now I love it so much.
[speaker001:] You really shouldn't, uh, te
[speaker005:] OK, everyone ready?
[speaker001:] You have to sort of, um [disfmarker] Jose, if you haven't done this, you have to plug your ears while you're t talking
[speaker002:] W wait [disfmarker] wait a minute [disfmarker] wait a minute. W we want [disfmarker] we want [disfmarker]
[speaker001:] so that you don't get confused, I guess.
[speaker002:] we want it synchronized.
[speaker008:] Yeah.
[speaker003:] Hey, you've done this before. Haven't you?
[speaker001:] Yeah. Oh, you've done this one before?
[speaker004:] That's [disfmarker]
[speaker003:] You've read [pause] digits together with us, haven't you [disfmarker]
[speaker001:] Together?
[speaker003:] I mean, at the same time?
[speaker008:] No.
[speaker003:] Oh, you haven't!
[speaker001:] I'm not [disfmarker] we [disfmarker] we [disfmarker] Oh, and you haven't done this either.
[speaker002:] OK.
[speaker003:] Oh, OK.
[speaker004:] Oh, yeah.
[speaker001:] I the first time is [pause] traumatic,
[speaker002:] We Y [vocalsound] Yeah,
[speaker001:] but [disfmarker]
[speaker003:] Oh, and the groupings are important,
[speaker002:] bu
[speaker008:] Mmm.
[speaker003:] so yo you're supposed to pause between the groupings.
[speaker008:] The grouping.
[speaker002:] Yeah.
[speaker008:] Yeah.
[speaker002:] OK. So, uh [disfmarker]
[speaker006:] You mean that the [disfmarker] the grouping is supposed to be synchronized?
[speaker005:] Yeah, sure.
[speaker003:] No.
[speaker002:] No, no.
[speaker006:] No? No?
[speaker002:] Synchronized digits.
[speaker003:] No.
[speaker001:] That'd be good. We we'll give everybody the same sheet
[speaker006:] It's like a [disfmarker] like a Greek [disfmarker] like a Greek choir?
[speaker001:] but they say different [disfmarker]
[speaker006:] You know? Like [disfmarker]
[speaker002:] Yes.
[speaker005:] Hey, what a good idea.
[speaker006:] Yeah.
[speaker005:] We could do the same sheet for everyone. Have them all read them at once.
[speaker001:] Well, different digits
[speaker004:] Eh [disfmarker]
[speaker001:] but same groupings.
[speaker005:] Or [disfmarker] or just same digits.
[speaker001:] So they would all be [disfmarker]
[speaker005:] See if anyone notices.
[speaker001:] Yeah.
[speaker003:] Yeah. That'd be good.
[speaker002:] There's so many possibilities.
[speaker003:] And then [disfmarker] then we can sing them next time.
[speaker002:] Uh. OK, why don't we go? Uh, one two three [disfmarker] Go!
[speaker003:] OK. Mmm!
[speaker002:] And Andreas has the last word.
[speaker005:] Did you read it twice or what?
[speaker001:] He's try No, he's trying to get good recognition performance.
[speaker003:] He had the h
[speaker008:] Yeah. Yeah.
[speaker003:] He had the [disfmarker] the long form.
[speaker006:] No.
[speaker005:] And we're off.