[speaker005:] So. OK. Doesn't look like it crashed. That's great.
[speaker007:] So I think maybe what's causing it to crash is I keep starting it and then stopping it to see if it's working. And so I think starting it and then stopping it and starting it again causes it to crash. So, I won't do that anymore.
[speaker002:] And it looks like you've found a way of uh mapping the location to the [disfmarker] without having people have to give their names each time?
[speaker001:] Sounds like an initialization thing.
[speaker002:] I mean it's like you have the [disfmarker]
[speaker007:] No.
[speaker002:] So you know that [disfmarker] I mean, are you going to write down [pause] that I sat here?
[speaker007:] I'm gonna collect the digit forms and write it down.
[speaker002:] OK.
[speaker003:] Oh, OK.
[speaker007:] So [disfmarker] So they should be right with what's on the digit forms. OK, so I'll go ahead and start with digits. u And I should say that uh, you just pau you just read each line an and then pause briefly.
[speaker005:] And start by giving the transcript number.
[speaker004:] Transcript [disfmarker] Uh. OK, OK.
[speaker001:] Tran Oh sorry, go ahead.
[speaker005:] So uh, you see, Don, the unbridled excitement of the work that we have on this project.
[speaker008:] OK.
[speaker005:] It's just uh [disfmarker]
[speaker008:] Umh.
[speaker005:] Uh, you know, it doesn't seem like a bad idea to have [comment] that information.
[speaker007:] And I'm surprised I sort of [disfmarker] I'm surprised I forgot that, but uh I think that would be a good thing to add.
[speaker005:] Yeah, I [disfmarker] I'd [disfmarker] I think it's some
[speaker007:] After I just printed out a zillion of them.
[speaker005:] Yeah, well, that's [disfmarker] Um, so I [disfmarker] I do have a [disfmarker] a an agenda suggestion. Uh, we [disfmarker] I think the things that we talk about in this meeting uh tend to be a mixture of uh procedural uh mundane things and uh research points and um I was thinking I think it was a meeting a couple of weeks ago that we [disfmarker] we spent much of the time talking about the mundane stuff cuz that's easier to get out of the way and then we sort of drifted into the research and maybe five minutes into that Andreas had to leave. So [vocalsound] uh I'm suggesting we turn it around and [disfmarker] and uh sort of we have [disfmarker] anybody has some mundane points that we could send an email later, uh hold them for a bit, and let's talk about the [disfmarker] the research y kind of things. Um, so um the one th one thing I know that we have on that is uh we had talked a [disfmarker] a couple weeks before um uh about the uh [disfmarker] the stuff you were doing with [disfmarker] with uh um uh l l attempting to locate events, we had a little go around trying to figure out what you meant by "events" but I think, you know, what we had meant by "events" I guess was uh points of overlap between speakers. But I th I gather from our discussion a little earlier today that you also mean uh interruptions with something else like some other noise.
[speaker004:] Yeah.
[speaker005:] Yes?
[speaker004:] Uh huh. Yeah.
[speaker005:] You mean that as an event also. So at any rate you were [disfmarker] you've [disfmarker] you've done some work on that
[speaker004:] To right.
[speaker005:] and um then the other thing would be it might be nice to have a preliminary discussion of some of the other uh research uh areas that uh we're thinking about doing. Um, I think especially since you [disfmarker] you haven't been in [disfmarker] in these meetings for a little bit, maybe you have some discussion of some of the p the plausible things to look at now that we're starting to get data, uh and one of the things I know that also came up uh is some discussions that [disfmarker] that uh [disfmarker] that uh Jane had with Lokendra uh about some [disfmarker] some [disfmarker] some um uh work about I [disfmarker] I [disfmarker] I d I [disfmarker] I don't want to try to say cuz I [disfmarker] I'll say it wrong, but anyway some [disfmarker] some potential collaboration there about [disfmarker] about the [disfmarker] about the [disfmarker] working with these data.
[speaker003:] Oh.
[speaker005:] So.
[speaker003:] Sure.
[speaker005:] So, uh.
[speaker007:] You wanna just go around?
[speaker005:] Uh. [pause] Well, I don't know if we [disfmarker] if this is sort of like everybody has something to contribute sort of thing, I think there's just just a couple [disfmarker] a couple people primarily um but um Uh, wh why don't [disfmarker] Actually I think that [disfmarker] that last one I just said we could do fairly quickly so why don't you [disfmarker] you start with that.
[speaker002:] OK. Shall I [disfmarker] shall I just start?
[speaker005:] Yeah, just explain what it was.
[speaker002:] OK. Um, so, uh, he was interested in the question of [disfmarker] you know, relating to his [disfmarker] to the research he presented recently, um of inference structures, and uh, the need to build in, um, this [disfmarker] this sort of uh mechanism for understanding of language. And he gave the example in his talk about how [pause] um, e a I'm remembering it just off the top of my head right now, but it's something about how um, i "Joe slipped" you know, "John had washed the floor" or something like that. And I don't have it quite right, but that kind of thing, where you have to draw the inference that, OK, there's this time sequence, but also the [disfmarker] the [disfmarker] the causal aspects of the uh floor and [disfmarker] and how it might have been the cause of the fall and that um it was the other person who fell than the one who cleaned it and it [disfmarker] [comment] These sorts of things. So, I looked through the transcript that we have so far, [comment] and um, fou identified a couple different types of things of that type and um, one of them was something like uh, during the course of the transcript, um um, w we had gone through the part where everyone said which channel they were on and which device they were on, and um, the question was raised "Well, should we restart the recording at this point?" And [disfmarker] and Dan Ellis said, "Well, we're just so far ahead of the game right now [pause] we really don't need to". Now, how would you interpret that without a lot of inference? So, the inferences that are involved are things like, OK, so, how do you interpret "ahead of the game"? You know. So it's the [disfmarker] it's [pause] i What you [disfmarker] what you int what you draw [disfmarker] you know, the conclusions that you need to draw are that space is involved in recording,
[speaker007:] Hmm, metaphorically.
[speaker002:] that um, i that [pause] i we have enough space, and he continues, like "we're so ahead of the game cuz now we have built in downsampling". So you have to sort of get the idea that um, "ahead of the game" is sp speaking with respect to space limitations, that um that in fact downsampling is gaining us enough space, and that therefore we can keep the recording we've done so far. But there are a lot of different things like that.
[speaker007:] So, do you think his interest is in using this as [pause] a data source, or [pause] training material, or what?
[speaker005:] Well, I [disfmarker] I should maybe interject to say this started off with a discussion that I had with him, so um we were trying to think of ways that his interests could interact with ours
[speaker007:] Mm hmm.
[speaker005:] and um uh I thought that if we were going to project into the future when we had a lot of data, uh and um such things might be useful for that in or before we invested too much uh effort into that he should uh, with Jane's help, look into some of the data that we're [disfmarker] already have
[speaker007:] Mm hmm.
[speaker005:] and see, is there anything to this at all? Is there any point which you think that, you know, you could gain some advantage and some potential use for it. Cuz it could be that you'd look through it and you say "well, this is just the wrong [pause] task for [disfmarker] for him to pursue his [disfmarker]"
[speaker007:] Wrong, yeah.
[speaker005:] And [disfmarker] and uh I got the impression from your mail that in fact there was enough things like this just in the little sample that [disfmarker] that you looked at that [disfmarker] that it's plausible at least.
[speaker002:] It's possible. Uh, he was [disfmarker] he [disfmarker] he [disfmarker] you know [disfmarker] We met and he was gonna go and uh you know, y look through them more systematically
[speaker005:] Yeah.
[speaker002:] and then uh meet again.
[speaker005:] Yeah.
[speaker002:] So it's, you know, not a matter of a [disfmarker]
[speaker005:] Yeah.
[speaker002:] But, yeah, I think [disfmarker] I think it was optimistic.
[speaker005:] So anyway, that's [disfmarker] that's e a quite different thing from anything we've talked about that, you know, might [disfmarker] might [disfmarker] might come out from some of this.
[speaker003:] But he can use text, basically. I mean, he's talking about just using text
[speaker002:] That's his major [disfmarker] I mentioned several that w had to do with implications drawn from intonational contours
[speaker003:] pretty much, or [disfmarker]?
[speaker002:] and [pause] that wasn't as directly relevant to what he's doing.
[speaker003:] OK.
[speaker002:] He's interested in these [disfmarker] these knowledge structures,
[speaker004:] Yeah, interesting.
[speaker002:] inferences that you draw [pause] i from [disfmarker]
[speaker005:] I mean, he certainly could use text, but we were in fact looking to see if there [disfmarker] is there [disfmarker] is there something in common between our interest in meetings and his interest in [disfmarker] in [disfmarker] in this stuff. So.
[speaker007:] And I imagine that transcripts of speech [disfmarker] I mean text that is speech [disfmarker] probably has more of those than sort of prepared writing. I [disfmarker] I don't know whether it would or not, but it seems like it would.
[speaker005:] I don't know, probably de probably depends on what the prepared writing was. But.
[speaker002:] Yeah, I don't think I would make that leap, because i in narratives, you know [disfmarker] I mean, if you spell out everything in a narrative, it can be really tedious,
[speaker007:] Mm hmm.
[speaker002:] so.
[speaker007:] Yeah, I'm just thinking, you know, when you're [disfmarker] when you're face to face, you have a lot of backchannel and [disfmarker] And [disfmarker]
[speaker002:] Oh. That aspect.
[speaker007:] Yeah. And so I think it's just easier to do that sort of broad inference jumping if it's face to face. I mean, so, if I just read that Dan was saying "we're ahead of the game" [comment] in that [disfmarker] in that context,
[speaker002:] Well [disfmarker] Yeah.
[speaker007:] I might not realize that he was talking about disk space as opposed to anything else.
[speaker002:] I [disfmarker] you know, I [disfmarker] I had several that had to do with backchannels and this wasn't one of them.
[speaker007:] Uh huh.
[speaker002:] This [disfmarker] this one really does um m make you leap from [disfmarker] So he said, you know, "we're ahead of the game, w we have built in downsampling".
[speaker007:] Mm hmm.
[speaker002:] And the inference, i if you had it written down, would be [disfmarker]
[speaker007:] I guess it would be the same.
[speaker002:] Uh huh. But there are others that have backchannelling, it's just he was less interested in those.
[speaker006:] Can I [disfmarker] Sorry to interrupt. Um, I f f f I've [disfmarker] [@ @] [comment] d A minute [disfmarker] uh, several minutes ago, I, like, briefly was [disfmarker] was not listening and [disfmarker] So who is "he" in this context?
[speaker003:] Yeah, there's a lot of pronoun [disfmarker]
[speaker006:] OK. So I was just realizing we've [disfmarker] You guys have been talking about "he" um for at least uh, I don't know, three [disfmarker] three four minutes without ever mentioning the person's name again.
[speaker003:] I believe it. Yeah. Actually to make it worse, [comment] uh, Morgan uses "you" and "you"
[speaker006:] So this is [disfmarker] this is [disfmarker] this is [disfmarker] gonna be a big, big problem if you want to later do uh, you know, indexing, or speech understanding of any sort.
[speaker007:] It's in my notes.
[speaker003:] with gaze and no identification, or [disfmarker] I just wrote this down.
[speaker006:] You just wrote this?
[speaker003:] Yeah, actually. Cuz Morgan will say well, "you had some ideas"
[speaker004:] Yeah.
[speaker003:] and he never said Li He looked [disfmarker]
[speaker007:] Well, I think he's doing that intentionally,
[speaker003:] Right,
[speaker007:] aren't you?
[speaker003:] so it's great.
[speaker006:] Right.
[speaker003:] So this is really great
[speaker004:] Yeah.
[speaker003:] because the thing is, because he's looking at the per even for addressees in the conversation,
[speaker006:] Mm hmm.
[speaker003:] I bet you could pick that up in the acoustics. Just because your gaze is also correlated with the directionality of your voice.
[speaker005:] Uh huh. Could be. Yeah. That would be tou
[speaker007:] Oh, that would be interesting.
[speaker002:] Can we
[speaker004:] Yeah.
[speaker003:] Yeah, so that, I mean, to even know um when [disfmarker] Yeah, if you have the P Z Ms you should be able to pick up what a person is looking at from their voice.
[speaker007:] Well, especially with Morgan, with the way we have the microphones arranged. I'm sort of right on axis and it would be very hard to tell.
[speaker003:] Right.
[speaker007:] Uh.
[speaker002:] Oh, but you'd have the [disfmarker]
[speaker003:] Put Morgan always like this and [disfmarker]
[speaker002:] You'd have fainter [disfmarker]
[speaker005:] Well, these [disfmarker]
[speaker002:] Wouldn't you get fainter reception out here?
[speaker007:] Sure, but I think if I'm talking like this? Right now I'm looking at Jane and talking, now I'm looking at Chuck and talking, I don't think the microphones would pick up that difference.
[speaker003:] But you don't have this [disfmarker] this problem.
[speaker002:] I see.
[speaker003:] Morgan is the one who does this most.
[speaker007:] So if I'm talking at you, or I'm talking at you.
[speaker005:] I probably been affect No, I th I think I've been affected by too many conversations where we were talking about lawyers and talking about [disfmarker] and concerns about "oh gee is somebody going to say something bad?" and so on.
[speaker007:] Lawyers.
[speaker005:] And so I [disfmarker] so I'm [disfmarker] I'm tending to stay away from people's names even though uh [disfmarker]
[speaker002:] I am too.
[speaker003:] Even though you could pick up later on, just from the acoustics who you were t who you were looking at.
[speaker002:] I am too.
[speaker007:] And we did mention who "he" was.
[speaker003:] Yeah.
[speaker007:] Early in the conversation.
[speaker005:] Yeah.
[speaker006:] Right, but I missed it. But [disfmarker] it was uh [disfmarker]
[speaker007:] Do [disfmarker] Sh Can I say
[speaker005:] Yeah.
[speaker003:] Yeah, yeah.
[speaker007:] or [disfmarker] or is that just too sensitive?
[speaker006:] Yeah.
[speaker005:] Yeah. No no, there's [disfmarker] No no, it isn't sensitive at all.
[speaker002:] Well [disfmarker]
[speaker005:] I was just [disfmarker] I was just [disfmarker] I was overreacting just because we've been talking about it.
[speaker002:] And in fact, it is [disfmarker] it is [disfmarker] it is sensitive.
[speaker005:] It's OK to [disfmarker]
[speaker003:] No, but that [disfmarker] it's interesting.
[speaker002:] I [disfmarker] I came up with something from the Human Subjects people that I wanted to mention. I mean, it fits into the m area of the mundane, but they did say [disfmarker] You know, I asked her very specifically about this clause of how, um, you know, it says "no individuals will be identified uh," in any publication using the data. "OK, well, individuals being identified, let's say you have a [disfmarker] a snippet that says," Joe s uh thinks such and such about [disfmarker] about this field, but I think he's wrongheaded. "Now I mean, we're [disfmarker] we're gonna be careful not to have the" wrongheaded "part in there, but [disfmarker] but you know, let's say we say, you know," Joe used to think so and so about this area, in his publication he says that but I think he's changed his mind. or whatever.
[speaker005:] b But I [disfmarker]
[speaker002:] Then the issue of [disfmarker] of being able to trace Joe, because we know he's well known in this field, and all this and [disfmarker] and tie it to the speaker, whose name was just mentioned a moment ago, can be sensitive. So I think it's really [disfmarker] really kind of adaptive and wise to not mention names any more than we have to because if there's a slanderous aspect to it, then how much to we wanna be able to have to remove?
[speaker005:] Yeah, well, there's that. But I [disfmarker] I mean I think also to some extent it's just educating the Human Subjects people, in a way, because there's [disfmarker] If uh [disfmarker] You know, there's court transcripts, there's [disfmarker] there's transcripts of radio shows [disfmarker] I mean people say people's names all the time. So I think it [disfmarker] it can't be bad to say people's names. It's just that [disfmarker] i I mean you're right that there's more poten If we never say anybody's name, then there's no chance of [disfmarker] of [disfmarker] of slandering anybody,
[speaker003:] But, then it won't [disfmarker] I mean, if we [disfmarker] if we [disfmarker]
[speaker005:] but [disfmarker]
[speaker007:] It's not a meeting.
[speaker003:] Yeah. I mean we should do whatever's natural in a meeting if [disfmarker] if we weren't being recorded.
[speaker005:] Yeah. Right, so I [disfmarker] So my behavior is probably not natural. So.
[speaker003:] "If Person X [disfmarker]"
[speaker002:] Well, my feeling on it was that it wasn't really important who said it, you know.
[speaker005:] Yeah.
[speaker006:] Well, if you ha since you have to um go over the transcripts later anyway, you could make it one of the jobs of the [pause] people who do that to mark
[speaker007:] Well, we t we t we talked about this during the anon anonymization. If we wanna go through and extract from the audio and the written every time someone says a name.
[speaker006:] Right.
[speaker007:] And I thought that our conclusion was that we didn't want to do that.
[speaker005:] Yeah, we really can't. But a actually, I'm sorry. I really would like to push [disfmarker] finish this off.
[speaker002:] I understand. No I just [disfmarker] I just was suggesting that it's not a bad policy p potentially.
[speaker005:] So it's [disfmarker]
[speaker002:] So, we need to talk about this later.
[speaker005:] Yeah, I di I didn't intend it an a policy though. It was [disfmarker] it was just it was just unconscious [disfmarker] well, semi conscious behavior.
[speaker002:] Uh huh.
[speaker005:] I sorta knew I was doing it but it was [disfmarker]
[speaker006:] Well, I still don't know who "he" is.
[speaker005:] I [disfmarker] I do I don't remember who "he" is.
[speaker003:] No, you have to say, you still don't know who "he" is, with that prosody.
[speaker005:] Ah. Uh, we were talking about Dan at one point [comment] and we were talking about Lokendra at another point.
[speaker006:] Oh.
[speaker002:] Yeah, depends on which one you mean.
[speaker005:] And I don't [disfmarker] I don't remember which [disfmarker] which part.
[speaker003:] It's ambiguous, so it's OK.
[speaker005:] Uh, I think [disfmarker]
[speaker007:] Well, the inference structures was Lokendra.
[speaker006:] But no. The inference stuff was [disfmarker] was [disfmarker] was Lokendra. OK.
[speaker005:] Yeah. Yeah. Yeah.
[speaker006:] That makes sense, yeah.
[speaker005:] Um [disfmarker]
[speaker003:] And the downsampling must have been Dan.
[speaker007:] Yeah.
[speaker005:] Good [disfmarker] Yeah.
[speaker003:] It's an inference.
[speaker005:] Yeah, you could do all these inferences,
[speaker007:] Yeah.
[speaker005:] yeah. Yeah. Um, I [disfmarker] I would like to move it into [disfmarker] into uh what Jose uh has been doing because he's actually been doing something.
[speaker002:] Yeah.
[speaker004:] Uh huh.
[speaker005:] So. [vocalsound] Right.
[speaker004:] OK.
[speaker006:] As opposed to the rest of us.
[speaker004:] Well [comment] [vocalsound] OK. I [disfmarker] I remind that me [disfmarker] my first objective eh, in the project is to [disfmarker] to study difference parameters to [disfmarker] to find a [disfmarker] a good solution to detect eh, the overlapping zone in eh speech recorded. But eh, [vocalsound] tsk, [comment] [vocalsound] ehhh [comment] In that way [comment] I [disfmarker] [vocalsound] I [disfmarker] [vocalsound] I begin to [disfmarker] to study and to analyze the ehn [disfmarker] the recorded speech eh the different session to [disfmarker] to find and to locate and to mark eh the [disfmarker] the different overlapping zone. And eh so eh I was eh [disfmarker] I am transcribing the [disfmarker] the first session and I [disfmarker] I have found eh, eh one thousand acoustic events, eh besides the overlapping zones, eh I [disfmarker] I [disfmarker] I mean the eh breaths eh aspiration eh, eh, talk eh, eh, clap, eh [disfmarker] [comment] I don't know what is the different names eh you use to [disfmarker] to name the [disfmarker] the [pause] n speech
[speaker001:] Nonspeech sounds?
[speaker004:] Yeah.
[speaker007:] Oh, I don't think we've been doing it at that level of detail. So.
[speaker004:] Yeah. Eh, [vocalsound] I [disfmarker] I [disfmarker] I do I don't need to [disfmarker] to [disfmarker] to mmm [vocalsound] [disfmarker] to m to label the [disfmarker] the different acoustic, but I prefer because eh I would like to [disfmarker] to study if eh, I [disfmarker] I will find eh, eh, a good eh parameters eh to detect overlapping I would like to [disfmarker] to [disfmarker] to test these parameters eh with the [disfmarker] another eh, eh acoustic events, to nnn [disfmarker] [vocalsound] to eh [disfmarker] to find what is the ehm [disfmarker] the false [disfmarker] eh, the false eh hypothesis eh, nnn, which eh are produced when we use the [disfmarker] the ehm [disfmarker] this eh parameter [disfmarker] eh I mean pitch eh, eh, difference eh, feature [disfmarker]
[speaker007:] Mm hmm.
[speaker001:] You know [disfmarker] I think some of these um that are the nonspeech overlapping events may be difficult even for humans to tell that there's two there.
[speaker007:] So it was [disfmarker]
[speaker004:] Yeah.
[speaker001:] I mean, if it's a tapping sound, you wouldn't necessarily [disfmarker] or, you know, something like that, it'd be [disfmarker] it might be hard to know that it was two separate events.
[speaker004:] Yeah. Yeah. Yeah. Yeah.
[speaker007:] Well [disfmarker] You weren't talking about just overlaps
[speaker004:] Ye
[speaker007:] were you? You were just talking about acoustic events.
[speaker004:] I [disfmarker] I [disfmarker] I [disfmarker] I t I t I talk eh about eh acoustic events in general,
[speaker007:] Someone starts, someone stops [disfmarker] Yeah.
[speaker004:] but eh my [disfmarker] my objective eh will be eh to study eh overlapping zone.
[speaker001:] Oh.
[speaker007:] Mm hmm.
[speaker004:] Eh? [comment] n Eh in twelve minutes I found eh, eh one thousand acoustic events.
[speaker005:] How many overlaps were there uh in it? No no, how many of them were the overlaps of speech, though?
[speaker004:] How many? Eh almost eh three hundred eh in one session
[speaker007:] Oh, God!
[speaker004:] in five [disfmarker] eh in forty five minutes.
[speaker007:] Ugh.
[speaker001:] Three hundred overlapping speech [disfmarker]
[speaker004:] Alm Three hundred overlapping zone.
[speaker003:] Overlapping speech.
[speaker004:] With the overlapping zone, overlapping speech [disfmarker] speech what eh different duration.
[speaker001:] Mm hmm.
[speaker005:] Sure.
[speaker002:] Does this [disfmarker]? So if you had an overlap involving three people, how many times was that counted?
[speaker004:] Yeah, three people, two people. Eh, um I would like to consider eh one people with difference noise eh in the background, be
[speaker005:] No no, but I think what she's asking is [pause] if at some particular for some particular stretch you had three people talking, instead of two, did you call that one event?
[speaker004:] Oh. Oh. Yeah. I consider one event eh for th for that eh for all the zone. This [disfmarker] th I [disfmarker] I [disfmarker] I con I consider [disfmarker] I consider eh an acoustic event, the overlapping zone, the period where three speaker or eh [disfmarker] are talking together.
[speaker007:] Well [disfmarker] So let's [disfmarker]
[speaker002:] For
[speaker007:] So let's say me and Jane are talking at the same time, and then Liz starts talking also over all of us. How many events would that be?
[speaker004:] So I don't understand.
[speaker007:] So, two people are talking, [comment] and then a third person starts talking.
[speaker004:] Yeah?
[speaker007:] Is there an event right here?
[speaker004:] Eh no. No no. For me is the overlapping zone, because [disfmarker] because you [disfmarker] you have s you have more one [disfmarker] eh, more one voice eh, eh produced in a [disfmarker] in [disfmarker] in a moment.
[speaker007:] So i if two or more people are talking.
[speaker005:] I see. OK. Yeah. So I think [disfmarker] Yeah. We just wanted to understand how you're defining it.
[speaker004:] Yeah.
[speaker005:] So then, in the region between [disfmarker] since there [disfmarker] there is some continuous region, in between regions where there is only one person speaking.
[speaker004:] If Uh huh.
[speaker005:] And one contiguous region like that you're calling an event. Is it [disfmarker] Are you calling the beginning or the end of it the event,
[speaker004:] Uh huh. Yeah.
[speaker005:] or are you calling the entire length of it the event?
[speaker004:] I consider the [disfmarker] the, nnn [disfmarker] the nnn, nnn [disfmarker] eh, the entirety eh, eh, all [disfmarker] all the time there were [disfmarker] the voice has overlapped.
[speaker005:] OK.
[speaker004:] This is the idea. But eh I [disfmarker] I don't distinguish between the [disfmarker] the numbers of eh speaker. Uh, I'm not considering [vocalsound] eh the [disfmarker] the [disfmarker] ehm [vocalsound] eh, the fact of eh, eh, for example, what did you say? Eh at first eh, eh two talkers are uh, eh speaking, and eh, eh a third person eh join to [disfmarker] to that. For me, it's eh [disfmarker] it's eh, all overlap zone, with eh several numbers of speakers is eh, eh the same acoustic event. Wi but [disfmarker] uh, without any mark between the zone [disfmarker] of the overlapping zone with two speakers eh speaking together, and the zone with the three speakers.
[speaker002:] That would j just be one.
[speaker004:] It [disfmarker] One. One.
[speaker002:] OK.
[speaker004:] Eh, with eh, a beginning mark and the ending mark.
[speaker005:] Got it.
[speaker004:] Because eh [vocalsound] for me, is the [disfmarker] is the zone with eh some kind of eh distortion the spectral. I don't mind [disfmarker] By the moment, by the moment.
[speaker007:] Well, but [disfmarker] But you could imagine that three people talking has a different spectral characteristic than two.
[speaker004:] I [disfmarker] I don't [disfmarker] Yeah, but eh [disfmarker] but eh I have to study. [comment] What will happen in a general way,
[speaker005:] Could.
[speaker007:] So. You had to start somewhere.
[speaker005:] Yeah. We just w
[speaker004:] I [disfmarker] [vocalsound] I don't know what eh will [disfmarker] will happen with the [disfmarker]
[speaker003:] So there's a lot of overlap.
[speaker007:] Yep.
[speaker003:] So.
[speaker007:] That's a lot of overlap, yeah,
[speaker005:] So again, that's [disfmarker] that's three [disfmarker] three hundred in forty five minutes that are [disfmarker] that are speakers, just speakers.
[speaker007:] for forty five minutes.
[speaker004:] Yeah? Yeah. Yeah.
[speaker005:] Uh huh. OK. Yeah.
[speaker002:] But a [disfmarker] a [disfmarker] a th
[speaker005:] So that's about eight per minute.
[speaker002:] But a thousand events in twelve minutes, that's [disfmarker]
[speaker004:] Yeah, [pause] but [disfmarker] Yeah.
[speaker003:] But that can include taps.
[speaker005:] Uh. Yeah.
[speaker004:] But [disfmarker]
[speaker002:] Well, but a thousand taps in eight minutes is a l in twelve minutes is a lot.
[speaker004:] General.
[speaker003:] Actually [disfmarker]
[speaker004:] I [disfmarker] I con I consider [disfmarker] I consider acoustic events eh, the silent too.
[speaker002:] Silent.
[speaker007:] Silence starting or silence ending [disfmarker]
[speaker004:] Yeah, silent, ground to [disfmarker] bec to detect [disfmarker] eh because I consider acoustic event all the things are not eh speech.
[speaker003:] Oh, OK.
[speaker005:] Mm hmm.
[speaker001:] Oh.
[speaker004:] In ge in [disfmarker] in [disfmarker] in a general point of view.
[speaker003:] Oh.
[speaker005:] OK, so how many of those thousand were silence?
[speaker003:] Alright.
[speaker006:] Not speech [disfmarker] not speech or too much speech.
[speaker004:] in the per Too much speech.
[speaker005:] Right. So how many of those thousand were silence, silent sections?
[speaker004:] Yeah. Uh silent, I [disfmarker] I [disfmarker] I [disfmarker] I don't [disfmarker] I [disfmarker] I haven't the [disfmarker] eh I [disfmarker] I would like to [disfmarker] to do a stylistic study
[speaker005:] Yeah.
[speaker004:] and give you eh with the report eh from eh the [disfmarker] the study from the [disfmarker] the [disfmarker] the session [disfmarker] one session.
[speaker005:] Yeah. Yeah.
[speaker004:] And I [disfmarker] I found that eh another thing. When eh [vocalsound] eh I w I [disfmarker] [vocalsound] I was eh look at eh nnn, the difference speech file, um, for example, eh if eh we use the ehm [disfmarker] the mixed file, to [disfmarker] to transcribe, the [disfmarker] the events and the words, I [disfmarker] I saw that eh the eh speech signal, collected by the eh this kind of mike [disfmarker] eh of this kind of mike, eh are different from the eh mixed signal eh, we eh [disfmarker] collected by headphone.
[speaker007:] Yep. Right.
[speaker004:] And [disfmarker] It's right.
[speaker005:] Yeah.
[speaker004:] But the problem is [vocalsound] the following. The [disfmarker] the [disfmarker] the [disfmarker] I [disfmarker] I [disfmarker] I knew that eh the signal eh, eh would be different, but eh the [disfmarker] the problem is eh, eh we eh detected eh difference events in the speech file eh collected by [disfmarker] by that mike uh qui compared with the mixed file. And so if [disfmarker] when you transcribe eh only eh using the nnn [disfmarker] the mixed file, it's possible [disfmarker] eh if you use the transcription to evaluate a different system, it's possible you eh [disfmarker] in the eh i and you use the eh speech file collected by the eh fet mike, to eh [disfmarker] to nnn [disfmarker] to do the experiments [pause] with the [disfmarker] the system,
[speaker005:] Mm hmm.
[speaker007:] Right.
[speaker004:] its possible to evaluate eh, eh [disfmarker] or to consider eh acoustic events that [disfmarker] which you marked eh in the mixed file, but eh they don't appear in the eh speech signal eh collected by the [disfmarker] by the mike.
[speaker007:] Right. The [disfmarker] the reason that I generated the mixed file was for IBM to do word level transcription, not speech event transcription.
[speaker004:] Yeah. Yeah. Oh, it's a good idea. It's a good idea I think.
[speaker007:] So I agree that if someone wants to do speech event transcription, that the mixed signals here [disfmarker]
[speaker004:] Yeah.
[speaker007:] I mean, if I'm tapping on the table, you it's not gonna show up on any of the mikes, but it's gonna show up rather loudly in the PZM.
[speaker004:] Yeah. Yeah. Yeah. So and I [disfmarker] I [disfmarker] [vocalsound] I say eh that eh, eh, or this eh only because eh I c I [disfmarker] I [disfmarker] [vocalsound] in my opinion, it's necessary to eh [disfmarker] to eh [disfmarker] to put the transcription on the speech file, collected by the objective signal.
[speaker007:] So.
[speaker004:] I mean the [disfmarker] the [disfmarker] the signal collected by the [disfmarker] eh, the real mike in the future, in the prototype to [disfmarker] to eh correct the initial eh segmentation eh with the eh real speech
[speaker005:] Mm hmm. The [disfmarker] the [disfmarker] the far field, yeah.
[speaker004:] you have to [disfmarker] to analyze [disfmarker] you have to [disfmarker] to process. Because I [disfmarker] I found a difference.
[speaker005:] Yeah, well, just [disfmarker] I mean, just in that [disfmarker] that one s ten second, or whatever it was, example that Adam had that [disfmarker] that we [disfmarker] we passed on to others a few months ago, there was that business where I g I guess it was Adam and Jane were talking at the same time and [disfmarker] and uh, in the close talking mikes you couldn't hear the overlap, and in the distant mike you could. So yeah, it's clear that if you wanna study [disfmarker] if you wanna find all the places where there were overlap, it's probably better to use a distant mike.
[speaker006:] That's good.
[speaker005:] On the other hand, there's other phenomena that are going on at the same time for which it might be useful to look at the close talking mikes,
[speaker004:] Yeah.
[speaker005:] so it's [disfmarker]
[speaker003:] But why can't you use the combination of the close talking mikes, time aligned?
[speaker007:] If you use the combination of the close talking mikes, you would hear Jane interrupting me, but you wouldn't hear the paper rustling. And so if you're interested in [disfmarker]
[speaker003:] I [disfmarker] I mean if you're interested in speakers overlapping other speakers and not the other kinds of nonspeech, that's not a problem,
[speaker005:] Some [comment] of it's masking [disfmarker] masked.
[speaker004:] Yeah.
[speaker001:] Were you interrupting him or was he interrupting you?
[speaker005:] Right.
[speaker007:] Right.
[speaker003:] right?
[speaker007:] Although the other issue is that the [pause] mixed close talking mikes [disfmarker]
[speaker004:] Yeah.
[speaker007:] I mean, I'm doing weird normalizations and things like that.
[speaker003:] But it's known.
[speaker004:] Yeah.
[speaker007:] Yep.
[speaker003:] I mean, the normalization you do is over the whole conversation
[speaker007:] Right.
[speaker003:] isn't it, over the whole meeting.
[speaker007:] Yep.
[speaker003:] So if you wanted to study people overlapping people, that's not a problem.
[speaker004:] I [disfmarker] I [disfmarker] I think eh I saw the nnn [disfmarker] the [disfmarker] eh but eh I eh [disfmarker] I have eh any results. I [disfmarker] I [disfmarker] I saw the [disfmarker] the speech file collected by eh the fet mike, and eh eh signal eh to eh [disfmarker] to noise eh relation is eh low.
[speaker005:] Mm hmm.
[speaker004:] It's low. It's very low. You would comp if we compare it with eh the headphone.
[speaker007:] Yep.
[speaker004:] And I [disfmarker] I found that nnn [disfmarker] that eh, [vocalsound] ehm, pr probably,
[speaker007:] Did [disfmarker] Did you
[speaker004:] I'm not sure eh by the moment, but it's [disfmarker] it's probably that eh a lot of eh, [vocalsound] eh for example, in the overlapping zone, on eh [disfmarker] in [disfmarker] in several eh parts of the files where you [disfmarker] you can find eh, eh [vocalsound] eh, smooth eh eh speech eh from eh one eh eh talker in the [disfmarker] in the meeting,
[speaker005:] Mm hmm. Mm hmm.
[speaker004:] it's probably in [disfmarker] in that eh [disfmarker] in [disfmarker] in those files you [disfmarker] you can not find [disfmarker] you can not process because eh it's confused with [disfmarker] with noise.
[speaker005:] Mm hmm.
[speaker004:] And there are [vocalsound] a lot of I think. But I have to study with more detail. But eh my idea is to [disfmarker] to process only [pause] nnn, this eh [disfmarker] nnn, this kind of s of eh speech. Because I think it's more realistic. I'm not sure it's a good idea, but eh [disfmarker]
[speaker007:] Well, it's more realistic but it'll [disfmarker] it'll be a lot harder.
[speaker005:] No [disfmarker] i Well, it'd be hard, but on the other hand as you point out, if your [disfmarker] if i if [disfmarker] if your concern is to get uh the overlapping people [disfmarker] people's speech, you will [disfmarker] you will get that somewhat better.
[speaker004:] Yeah. Mm hmm. Yeah.
[speaker005:] Um, Are you making any use [disfmarker] uh you were [disfmarker] you were working with th the data that had already been transcribed. Does it uh [disfmarker]
[speaker004:] With [disfmarker] By Jane.
[speaker005:] Yes.
[speaker004:] Yeah.
[speaker005:] Now um did you make any use of that? See I was wondering cuz we st we have these ten hours of other stuff that is not yet transcribed.
[speaker004:] Yeah. Yeah.
[speaker005:] Do you [disfmarker]
[speaker004:] The [disfmarker] the transcription by Jane, t eh i eh, I [disfmarker] I [disfmarker] I want to use to [disfmarker] to nnn, [vocalsound] eh to put [disfmarker] i i it's a reference for me. But eh the transcription [disfmarker] eh for example, I [disfmarker] I don't [disfmarker] I [disfmarker] I'm not interested in the [disfmarker] in the [disfmarker] in the words, transcription words, eh transcribed eh eh in [disfmarker] eh follow in the [disfmarker] [vocalsound] in the [disfmarker] in the speech file, but eh eh Jane eh for example eh put a mark eh at the beginning eh of each eh talker, in the [disfmarker] in the meeting, um eh she [disfmarker] she nnn includes information about the zone where eh there are eh [disfmarker] there is an overlapping zone.
[speaker005:] Mm hmm.
[speaker004:] But eh there isn't any [disfmarker] any mark, time [disfmarker] temporal mark, to [disfmarker] to c eh [disfmarker] to mmm [vocalsound] [disfmarker] e heh, to label [comment] the beginning and the end of the [disfmarker] of the
[speaker005:] OK. Right, so she is [disfmarker]
[speaker004:] ta I'm [disfmarker] I [disfmarker] I [disfmarker] I think eh we need this information to
[speaker005:] Right. So the twelve [disfmarker] you [disfmarker] you [disfmarker] it took you twelve hours [disfmarker] of course this included maybe some [disfmarker] some time where you were learning about what [disfmarker] what you wanted to do, but [disfmarker] but uh, it took you something like twelve hours to mark the forty five minutes, your
[speaker007:] Twelve minutes.
[speaker004:] Twelve minutes.
[speaker005:] s Twelve minutes!
[speaker004:] Twelve minutes. Twelve.
[speaker005:] I thought you did forty five minutes of [disfmarker]
[speaker004:] No, forty five minutes is the [disfmarker] is the session, all the session.
[speaker005:] Oh, you haven't done the whole session.
[speaker002:] Oh.
[speaker004:] Yeah,
[speaker005:] This is just twelve minutes.
[speaker004:] all is the [vocalsound] the session.
[speaker005:] Oh.
[speaker004:] Tw twelve hours of work to [disfmarker] [vocalsound] to segment eh and label eh twelve minutes from a session of part [disfmarker] of f
[speaker005:] So [comment] let me back up again. So the [disfmarker] when you said there were three hundred speaker overlaps,
[speaker004:] Yeah.
[speaker005:] that's in twelve minutes?
[speaker004:] No no no. I [disfmarker] I consider all the [disfmarker] all the session because eh I [disfmarker] I count the nnn [disfmarker] the nnn [disfmarker] the overlappings marked by [disfmarker] by Jane,
[speaker005:] Oh, OK. OK.
[speaker002:] Oh, I see.
[speaker004:] in [disfmarker] in [disfmarker] in [disfmarker] in the [pause] fin in [disfmarker] in the [pause] forty five minutes.
[speaker005:] So it's three hundred in forty five minutes, but you have [disfmarker] you have time uh, uh marked [disfmarker] twelve minute [disfmarker] the [disfmarker] the [disfmarker] the um overlaps in twelve minutes of it.
[speaker004:] Yeah.
[speaker005:] Got it.
[speaker007:] Well, not just the overlaps, everything.
[speaker006:] So, can I ask [disfmarker] [vocalsound] can I ask whether you found [disfmarker] uh, you know, how accurate uh Jane's uh uh labels were as far as [disfmarker] you know, did she miss some overlaps? or did she n?
[speaker004:] But, by [disfmarker] by the moment, I [disfmarker] I don't compare, my [disfmarker] my temporal mark with eh Jane, but eh I [disfmarker] I want to do it. Because eh eh i per perhaps I have eh errors in the [disfmarker] in the marks, I [disfmarker] and if I [disfmarker] I compare with eh Jane, it's probably I [disfmarker] I [disfmarker] I can correct and [disfmarker] and [disfmarker] and [disfmarker] to get eh eh a more accurately eh eh transcription in the file.
[speaker005:] Yeah.
[speaker007:] Well, also Jane [disfmarker] Jane was doing word level.
[speaker005:] Yeah.
[speaker007:] So we weren't concerned with [comment] exactly when an overlap started and stopped.
[speaker004:] Yeah.
[speaker006:] Right. Right. I'm expect I'm not expecting [disfmarker]
[speaker004:] Well [disfmarker]
[speaker003:] Well, not only a word level, but actually I mean, you didn't need to show the exact point of interruption,
[speaker004:] No, it's [disfmarker]
[speaker003:] you just were showing at the level of the phrase or the level of the speech spurt, or [disfmarker]
[speaker007:] Right.
[speaker005:] Mm hmm.
[speaker007:] Yep.
[speaker004:] Yeah.
[speaker002:] Well [disfmarker]
[speaker004:] Yeah.
[speaker002:] Well, yeah, b yeah, I would say time bin. So my [disfmarker] my goal is to get words with reference to a time bin, [pause] beginning and end point.
[speaker003:] Yeah.
[speaker004:] Yeah. Yeah.
[speaker003:] Right.
[speaker004:] Yeah.
[speaker002:] And [disfmarker] and sometimes, you know, it was like you could have an overlap where someone said something in the middle, but, yeah, w it just wasn't important for our purposes to have it that [disfmarker] i disrupt that unit in order to have, you know, a the words in the order in which they were spoken,
[speaker004:] Yeah.
[speaker002:] it would have [disfmarker] it would have been hard with the interface that we have.
[speaker007:] Right.
[speaker002:] Now, my [disfmarker] a Adam's working on a of course, on a revised overlapping interface,
[speaker004:] Uh huh. I [disfmarker] I [disfmarker] I think [disfmarker] It's [disfmarker] it's a good eh work,
[speaker002:] but [disfmarker]
[speaker004:] but eh I think we need eh eh more information.
[speaker006:] No, of course. I expect you to find more overlaps than [disfmarker] than Jane
[speaker002:] Yeah.
[speaker007:] Always need more for [disfmarker]
[speaker004:] No, no. I [disfmarker] I have to go to [disfmarker]
[speaker002:] Yeah.
[speaker006:] because you're looking at it at a much more detailed level.
[speaker004:] I want eh [disfmarker] I wanted to eh compare the [disfmarker] the transcription.
[speaker007:] But if it takes sixty to one [disfmarker]
[speaker005:] I have [disfmarker] Well, I but I have a suggestion about that. Um, obviously this is very, very time consuming, and you're finding lots of things which I'm sure are gonna be very interesting, but in the interests of making progress, uh might I s how [disfmarker] how would it affect your time if you only marked speaker overlaps?
[speaker004:] Only.
[speaker005:] Yes. Do not mark any other events,
[speaker004:] Yeah.
[speaker005:] but only mark speaker [disfmarker]
[speaker004:] Uh huh.
[speaker005:] Do you think that would speed it up quite a bit?
[speaker004:] OK. OK. I [disfmarker] I [disfmarker] I [disfmarker] I w I [disfmarker] I wanted to [disfmarker]
[speaker005:] Do y do you think that would speed it up? Uh, speed up your [disfmarker] your [disfmarker] your marking?
[speaker004:] nnn, I don't understand very.
[speaker005:] It took you a long time [pause] to mark twelve minutes.
[speaker004:] Yeah. Oh, yeah, yeah.
[speaker005:] Now, my suggestion was for the other thirty three [disfmarker]
[speaker004:] On only to mark [disfmarker] only to mark overlapping zone, but [disfmarker]
[speaker005:] Yeah, and my question is, if you did that, if you followed my suggestion, would it take much less time?
[speaker004:] Oh, yeah.
[speaker005:] Yeah OK.
[speaker004:] Sure. Yeah sure.
[speaker005:] Then I think it's a good idea. Then I think it's a good idea, because it
[speaker004:] Sure sure. Sure, because I [disfmarker] I need a lot of time to [disfmarker] to put the label or to do that. Yeah.
[speaker005:] Yeah, I mean, we we know that there's noise.
[speaker007:] And
[speaker004:] Uh huh.
[speaker005:] There's [disfmarker] there's uh continual noise uh from fans and so forth, and there is uh more impulsive noise from uh taps and so forth
[speaker004:] Yeah.
[speaker005:] and [disfmarker] and something in between with paper rustling. We know that all that's there and it's a g worthwhile thing to study, but obviously it takes a lot of time to mark all of these things.
[speaker004:] Yeah.
[speaker005:] Whereas th i I would think that uh you [disfmarker] we can study more or less as a distinct phenomenon the overlapping of people talking.
[speaker004:] Uh huh. OK.
[speaker005:] So. Then you can get the [disfmarker] Cuz you need [disfmarker]
[speaker004:] OK.
[speaker005:] If it's three hundred uh [disfmarker] i i it sounds like you probably only have fifty or sixty or seventy events right now that are really [disfmarker]
[speaker004:] Yeah.
[speaker005:] And [disfmarker] and you need to have a lot more than that to have any kind of uh even visual sense of [disfmarker] of what's going on, much less any kind of reasonable statistics.
[speaker007:] Right.
[speaker003:] Now, why do you need to mark speaker overlap by hand if you can infer it from the relative energy in the [disfmarker]
[speaker007:] Well, that's [disfmarker] That's what I was gonna bring up.
[speaker003:] I mean, you shouldn't need to do this p completely by hand,
[speaker005:] Um, OK, yeah. So let's back up because you weren't here for an earlier conversation.
[speaker003:] right? I'm sorry.
[speaker005:] So the idea was that what he was going to be doing was experimenting with different measures such as the increase in energy, such as the energy in the LPC residuals, such as [disfmarker] I mean there's a bunch of things [disfmarker] I mean, increased energy is is sort of an obvious one.
[speaker003:] Mm hmm. In the far field mike.
[speaker005:] Yeah.
[speaker003:] Oh, OK.
[speaker005:] Um, and uh, it's not obvious, I mean, you could [disfmarker] you could do the dumbest thing and get [disfmarker] get it ninety percent of the time. But when you start going past that and trying to do better, it's not obvious what combination of features is gonna give you the [disfmarker] you know, the right detector. So the idea is to have some ground truth first. And so the i the idea of the manual marking was to say "OK this, i you know, it's [disfmarker] it's really here".
[speaker001:] But I think Liz is saying why not get it out of the transcripts?
[speaker003:] What I mean is [pause] get it from the close talking mikes.
[speaker005:] Uh, yeah.
[speaker003:] A or ge get a first pass from those,
[speaker005:] We t we t w we t we talked about that.
[speaker003:] and then go through sort of [disfmarker] It'd be a lot faster probably to [disfmarker]
[speaker006:] And you can [disfmarker]
[speaker007:] Yeah, that's his, uh [disfmarker]
[speaker005:] We [disfmarker] we [disfmarker] we talked about that. s But so it's a bootstrapping thing and the thing is,
[speaker003:] Yeah, I just [disfmarker]
[speaker005:] the idea was, i we i i we thought it would be useful for him to look at the data anyway, and [disfmarker] and then whatever he could mark would be helpful,
[speaker003:] Right.
[speaker005:] and we could [disfmarker] Uh it's a question of what you bootstrap from. You know, do you bootstrap from a simple measurement which is right most of the time and then you g do better, or do you bootstrap from some human being looking at it and then [disfmarker] then do your simple measurements, uh from the close talking mike. I mean, even with the close talking mike you're not gonna get it right all the time.
[speaker003:] Well, that's what I wonder, because um [disfmarker] or how bad it is,
[speaker005:] Well
[speaker007:] I'm working on a program to do that, and [disfmarker]
[speaker003:] be um, because that would be interesting especially because the bottleneck is the transcription. Right? I mean, we've got a lot more data than we have transcriptions for. We have the audio data,
[speaker005:] Yeah.
[speaker003:] we have the close talking mike, so I mean it seems like one kind of project that's not perfect, but [disfmarker] um, that you can get the training data for pretty quickly is, you know, if you infer form the close talking mikes where the on off points are of speech,
[speaker005:] Right, we discussed that.
[speaker003:] you know, how can we detect that from a far field?
[speaker007:] And [disfmarker]
[speaker002:] Oh.
[speaker007:] I've [disfmarker] I've written a program to do that,
[speaker003:] OK,
[speaker007:] and it, uh [disfmarker]
[speaker003:] I'm sorry I missed the [disfmarker]
[speaker005:] It's OK.
[speaker007:] and [disfmarker] so [disfmarker] but it's [disfmarker] it's doing something very, very simple. It just takes a threshold, based on [disfmarker] on the volume,
[speaker003:] Uh huh.
[speaker006:] Or you can set the threshold low and then weed out the false alarms by hand.
[speaker003:] Right, by hand. Yeah.
[speaker006:] Yeah.
[speaker007:] um, and then it does a median filter, and then it looks for runs. And, it seems to work, I've [disfmarker] I'm sort of fiddling with the parameters, to get it to actually generate something, and I haven't [disfmarker] I don't [disfmarker] what I'm working on [disfmarker] was working on [disfmarker] was getting it to a form where we can import it into the user interface that we have, [pause] into Transcriber. And so [disfmarker] I told [disfmarker] I said it would take about a day. I've worked on it for about half a day,
[speaker008:] I have to go.
[speaker007:] so give me another half day and I we'll have something we can play with.
[speaker003:] OK.
[speaker005:] See, this is where we really need the Meeting Recorder query stuff to be working, because we've had these meetings and we've had this discussion about this, and I'm sort of remembering a little bit about what we decided,
[speaker003:] Right. I'm sorry. I just [disfmarker]
[speaker005:] but I couldn't remember all of it.
[speaker003:] It
[speaker005:] So, I think it was partly that, you know, give somebody a chance to actually look at the data and see what these are like, partly that we have e some ground truth to compare against, you know, when [disfmarker] when he [disfmarker] he gets his thing going,
[speaker007:] But [disfmarker]
[speaker005:] uh, and [disfmarker]
[speaker003:] Well, it's definitely good to have somebody look at it. I was just thinking as a way to speed up you know, the amount of [disfmarker]
[speaker005:] That was [disfmarker] that was exactly the notion that [disfmarker] that [disfmarker] that we discussed.
[speaker002:] Mm hmm.
[speaker003:] OK.
[speaker007:] Thanks.
[speaker002:] Another thing we discussed was um that [disfmarker]
[speaker005:] So.
[speaker003:] It looks good. I'll be in touch. Thanks.
[speaker005:] S See ya. Yeah.
[speaker002:] Was that um there m [pause] there was this already a script I believe uh that Dan had written, [comment] that uh handle bleedthrough, I mean cuz you have this [disfmarker] this close [disfmarker] you have contamination from other people who speak loudly.
[speaker007:] Yeah, and I haven't tried using that. It would probably help the program that I'm doing to first feed it through that. It's a cross correlation filter. So I [disfmarker] I haven't tried that, but that [disfmarker] If [disfmarker] It [disfmarker] it might be something [disfmarker] it might be a good way of cleaning it up a little.
[speaker002:] So, some thought of maybe having [disfmarker] Yeah, having that be a preprocessor and then run it through yours.
[speaker007:] Exactly. Yep.
[speaker005:] But [disfmarker] but that's a refinement
[speaker002:] That's what we were discussing.
[speaker005:] and I think we wanna see [disfmarker] try the simple thing first, cuz you add this complex thing up uh afterwards that does something good y y yo you sort of wanna see what the simple thing does first.
[speaker007:] Yep.
[speaker005:] But uh, having [disfmarker] having somebody have some experience, again, with [disfmarker] with uh [disfmarker] with marking it from a human standpoint, we're [disfmarker] I mean, I don't expect Jose to [disfmarker] to do it for uh f fifty hours of [disfmarker] [comment] of speech, but I mean we [disfmarker] [comment] if uh [disfmarker] if he could speed up what he was doing by just getting the speaker overlaps so that we had it, say, for forty five minutes, then at least we'd have three hundred examples of it.
[speaker004:] Yeah. Sure. Sure.
[speaker005:] And when [disfmarker] when uh Adam was doing his automatic thing he could then compare to that and see what it was different.
[speaker003:] Oh yeah, definitely.
[speaker001:] You know, I did [disfmarker] I did uh something almost identical to this at one of my previous jobs, and it works pretty well. I mean, i almost exactly what you described, an energy detector with a median filter, you look for runs. And uh, you know, you can [disfmarker]
[speaker007:] It seemed like the right thing to do.
[speaker001:] Yeah. I mean, you [disfmarker] you can get y I mean, you get them pretty close.
[speaker007:] That was with zero literature search.
[speaker001:] And so I think doing that to generate these possibilities and then going through and saying yes or no on them would be a quick way to [disfmarker] to do it.
[speaker007:] That's good validation.
[speaker001:] Yeah.
[speaker002:] Is this proprietary?
[speaker007:] Yeah, do you have a patent on it?
[speaker001:] Uh. [comment] No. No. It was when I was working for the government.
[speaker005:] Oh, then everybody owns it. It's the people.
[speaker002:] Well, I mean, is this something that we could just co opt, or is it [disfmarker]? No.
[speaker001:] Nah.
[speaker002:] OK.
[speaker005:] Well, i i i he's pretty close, anyway. I think [disfmarker] I think it's [disfmarker]
[speaker001:] Yeah, he's [disfmarker] it [disfmarker] it doesn't take a long time.
[speaker002:] Right. I just thought if it was tried and true, then [disfmarker] [comment] and he's gone through additional levels of [disfmarker] of development.
[speaker007:] Just output. Although if you [disfmarker] if you have some parameters like what's a good window size for the median filter [disfmarker]
[speaker001:] Oh! [comment] I have to remember. I'll think about it, and try to remember.
[speaker006:] And it might be different for government people.
[speaker007:] That's alright.
[speaker005:] Yeah, good enough for government work, as they say.
[speaker003:] They [disfmarker] they [disfmarker]
[speaker001:] Di dif different [disfmarker] different bandwidth.
[speaker006:] They
[speaker007:] I was doing pretty short, you know, tenth of a second, [comment] sorts of numbers.
[speaker006:] OK.
[speaker005:] Uh, I don't know, it [disfmarker] if [disfmarker] if we want to uh [disfmarker] So, uh, maybe we should move on to other [disfmarker] other things in limited time.
[speaker002:] Can I ask one question about his statistics? So [disfmarker] so in the tw twelve minutes, um, if we took three hundred and divided it by four, which is about the length of twelve minutes, i Um, I'd expect like there should be seventy five overlaps.
[speaker005:] Yeah.
[speaker002:] Did you find uh more than seventy five overlaps in that period, or [disfmarker]?
[speaker004:] More than?
[speaker002:] More than [disfmarker] How many overlaps in your twelve minutes?
[speaker004:] How many? Eh, not [@ @] I Onl only I [disfmarker] I transcribe eh only twelve minutes from the
[speaker005:] Yeah.
[speaker004:] but eh I [disfmarker] I don't co eh [disfmarker] I don't count eh the [disfmarker] the overlap.
[speaker002:] The overlaps. OK.
[speaker004:] I consider I [disfmarker] I [disfmarker] The [disfmarker] the nnn [disfmarker] The [disfmarker] the three hundred is eh considered only you [disfmarker] your transcription. I have to [disfmarker] [vocalsound] to finish transcribing. So.
[speaker007:] I b I bet they're more, because the beginning of the meeting had a lot more overlaps than [disfmarker] than sort of the middle.
[speaker004:] Yeah.
[speaker007:] Middle or end.
[speaker004:] Yeah.
[speaker007:] Because i we're [disfmarker] we're dealing with the [disfmarker] Uh, in the early meetings,
[speaker002:] I'm not sure.
[speaker007:] we're recording while we're saying who's talking on what microphone, [comment] and things like that,
[speaker004:] Yeah.
[speaker007:] and that seems to be a lot of overlap.
[speaker004:] Yeah.
[speaker002:] I think it's an empirical question. I think we could find that out.
[speaker004:] Yeah.
[speaker007:] Yep.
[speaker002:] I'm [disfmarker] I'm not sure that the beginning had more.
[speaker005:] So [disfmarker] so I was gonna ask, I guess about any [disfmarker] any other things that [disfmarker] that [disfmarker] that either of you wanted to talk about, especially since Andreas is leaving in five minutes, that [disfmarker] that you wanna go with.
[speaker003:] Can I just ask about the data, like very straightforward question is where we are on the amount of data and the amount of transcribed data, just cuz I'm [disfmarker] I wanted to get a feel for that to sort of be able to know what [disfmarker] what can be done first and like how many meetings are we recording
[speaker005:] Right so there's this [disfmarker] this [disfmarker] There's this forty five minute piece that Jane transcribed.
[speaker003:] and [disfmarker]
[speaker005:] That piece was then uh sent to IBM so they could transcribe so we have some comparison point. Then there's s a larger piece that's been recorded and uh put on CD ROM and sent uh to IBM. Right? And then we don't know.
[speaker003:] How many meetings is that?
[speaker007:] What's that?
[speaker003:] Like [disfmarker] how many [disfmarker]
[speaker005:] That was about ten hours, and there was about [disfmarker]
[speaker003:] t ten [disfmarker] It's like ten meetings or something?
[speaker007:] Yeah, something like that.
[speaker003:] Uh huh.
[speaker007:] And then [disfmarker] then we
[speaker001:] Ten meetings that have been sent to IBM?
[speaker003:] And [disfmarker]
[speaker007:] Well, I haven't sent them yet because I was having this problem with the [pause] missing files.
[speaker005:] Yeah. Oh. Oh, that's right, that had [disfmarker] those have not been sent.
[speaker001:] H how many total have we recorded now, altogether?
[speaker005:] We're saying about [pause] twelve hours.
[speaker007:] About twelve [pause] by now. Twelve or thirteen.
[speaker003:] Uh huh. And we're recording only this meeting, like continuously we're only recording this one now? or [disfmarker]?
[speaker005:] No.
[speaker007:] Nope.
[speaker005:] No, so the [disfmarker] the [disfmarker] that's the [disfmarker] that's the biggest one [disfmarker] uh, chunk so far,
[speaker003:] OK.
[speaker001:] It was the morning one.
[speaker005:] but there's at least one meeting recorded of uh the uh uh natural language guys.
[speaker007:] Jerry.
[speaker003:] Do they meet every week,
[speaker005:] And then there [disfmarker] Uh, they do.
[speaker003:] or every [disfmarker]
[speaker005:] w w And we talked to them about recording some more and we're going to, uh, we've started having a morning meeting, today uh i starting a w a week or two ago, on the uh front end issues, and we're recording those, uh there's a network services and applications group here who's agreed to have their meetings recorded,
[speaker003:] Great.
[speaker005:] and we're gonna start recording them. They're [disfmarker] They meet on Tuesdays. We're gonna start recording them next week. So actually, we're gonna h start having a [disfmarker] a pretty significant chunk and so, you know, [vocalsound] Adam's sort of struggling with trying to get things to be less buggy, and come up quicker when they do crash and stuff [disfmarker] things like that, now that uh [disfmarker] [vocalsound] the things are starting to happen. So right now, yeah, I th I'd say the data is predominantly meeting meetings, but there are scattered other meetings in it and that [disfmarker] that amount is gonna grow uh so that the meeting meetings will probably ultimately [disfmarker] i if we're [disfmarker] if we collect fifty or sixty hours, the meeting meetings it will probably be, you know, twenty or thirty percent of it, not [disfmarker] not [disfmarker] not eighty or ninety. But.
[speaker003:] So there's probably [disfmarker] there's three to four a week,
[speaker007:] That's what we're aiming for.
[speaker003:] that we're aiming for.
[speaker005:] Yeah.
[speaker003:] And they're each about an hour or something.
[speaker005:] Yeah, yeah.
[speaker007:] Although [disfmarker] Yeah. We'll find out tomorrow whether we can really do this or not.
[speaker003:] So [disfmarker]
[speaker005:] Yeah and th the [disfmarker] the other thing is I'm not pos
[speaker003:] OK.
[speaker005:] I'm sort of thinking as we've been through this a few times, that I really don't know [disfmarker] maybe you wanna do it once for the novelty, but I don't know if in general we wanna have meetings that we record from outside this group do the digits.
[speaker007:] Right.
[speaker005:] Because it's just an added bunch of weird stuff.
[speaker003:] Yeah.
[speaker005:] And, you know, we [disfmarker] we h we're highly motivated. Uh in fact, the morning group is really motivated cuz they're working on connected digits, so it's [disfmarker]
[speaker007:] Actually that's something I wanted to ask, is I have a bunch of scripts to help with the transcription of the digits.
[speaker005:] Yeah.
[speaker007:] We don't have to hand transcribe the digits because we're reading them and I have those.
[speaker005:] Yeah.
[speaker007:] And so I have some scripts that let you very quickly extract the sections of each utterance.
[speaker003:] Right.
[speaker007:] But I haven't been ru I haven't been doing that. Um, if I did that, is someone gonna be working on it?
[speaker005:] Uh, yeah, I [disfmarker] I think
[speaker007:] I mean, is it something of interest?
[speaker005:] definitely s so Absolutely. Yeah, whoever we have working on the acoustics for the Meeting Recorder are gonna start with that.
[speaker007:] OK. I mean, I I'm [disfmarker] I'm interested in it, I just don't have time to do it now.
[speaker006:] I was [disfmarker] these meetings [disfmarker] I'm sure someone thought of this, but these [disfmarker] this uh reading of the numbers would be extremely helpful to do um adaptation.
[speaker007:] So Yep. Yep.
[speaker006:] Um.
[speaker003:] Actually I have o
[speaker007:] I [disfmarker] I would really like someone to do adaptation.
[speaker006:] Mm hmm.
[speaker007:] So if we got someone interested in that, I think it would be great for Meeting Recorder.
[speaker005:] Well [disfmarker] I mean, one of the things I wanted to do, uh, that I I talked to [disfmarker] to Don about, is one of the possible things he could do or m also, we could have someone else do it, is to do block echo cancellation,
[speaker007:] Since it's the same people over and over.
[speaker006:] Mm hmm.
[speaker005:] to try to get rid of some of the effects of the [disfmarker] the [disfmarker] the far field effects.
[speaker006:] Mm hmm.
[speaker005:] Um, I mean we have [disfmarker] the party line has been that echo cancellation is not the right way to handle the situation because people move around, and uh, if [disfmarker] if it's [disfmarker] if it's uh not a simple echo, like a cross talk kind of echo, but it's actually room acoustics, it's [disfmarker] it's [disfmarker] it's [disfmarker] you can't really do inversion,
[speaker006:] Mm hmm.
[speaker005:] and even echo cancellation is going to uh be something [disfmarker] It may [disfmarker] you [disfmarker] Someone may be moving enough that you are not able to adapt quickly and so the tack that we've taken is more "lets come up with feature approaches and multi stream approaches and so forth, that will be robust to it for the recognizer and not try to create a clean signal".
[speaker006:] Mm hmm.
[speaker005:] Uh, that's the party line. But it occurred to me a few months ago that uh party lines are always, you know, sort of dangerous. It's good [disfmarker] [vocalsound] good to sort of test them, actually. And so we haven't had anybody try to do a good serious job on echo cancellation and we should know how well that can do. So that's something I'd like somebody to do at some point, just take these digits, take the far field mike signal, and the close uh mike signal, and apply really good echo cancellation. Um, there was a [disfmarker] have been some nice talks recently by [disfmarker] by Lucent on [disfmarker] on their b
[speaker006:] Hmm.
[speaker005:] the block echo cancellation particularly appealed to me, uh you know, trying and change it sample by sample, but you have some reasonable sized blocks. [comment] And um, you know, th
[speaker001:] W what is the um [disfmarker] the artifact you try to [disfmarker] you're trying to get rid of when you do that?
[speaker006:] Ciao.
[speaker005:] Uh so it's [disfmarker] it [disfmarker] you have a [disfmarker] a direct uh [disfmarker] Uh, what's the difference in [disfmarker] If you were trying to construct a linear filter, that would um [disfmarker]
[speaker006:] I'm signing off.
[speaker005:] Yeah. that would subtract off [comment] the um uh parts of the signal that were the aspects of the signal that were different between the close talk and the distant. You know, so [disfmarker] so uh um I guess in most echo cancellation [disfmarker] Yeah, so you [disfmarker] Given that um [disfmarker] Yeah, so you're trying to [disfmarker] So you'd [disfmarker] There's a [disfmarker] a distance between the close and the distant mikes so there's a time delay there, and after the time delay, there's these various reflections. And if you figure out well what's the [disfmarker] there's a [disfmarker] a least squares algorithm that adjusts itself [disfmarker] adjusts the weight so that you try to subtract [disfmarker] essentially to subtract off uh different uh [disfmarker] different reflections. Right? So let's take the simple case where you just had [disfmarker] you had some uh some delay in a satellite connection or something and then there's a [disfmarker] there's an echo. It comes back. And you want to adjust this filter so that it will maximally reduce the effect of this echo.
[speaker001:] So that would mean like if you were listening to the data that was recorded on one of those. Uh, just the raw data, you would [disfmarker] you might hear kind of an echo? And [disfmarker] and then this [disfmarker] noise cancellation would get
[speaker005:] Well, I'm [disfmarker] I'm [disfmarker] I'm saying [disfmarker] That's a simplified version of what's really happening. [comment] What's really happening is [disfmarker] Well, when I'm talking to you right now, you're getting the direct sound from my speech, but you're also getting, uh, the indirect sound that's bounced around the room a number of times. OK? So now, if you um try to r you [disfmarker] To completely remove the effect of that is sort of impractical for a number of technical reasons, but I [disfmarker] but [disfmarker] not to try to completely remove it, that is, invert the [disfmarker] the room response, but just to try to uh uh eliminate some of the [disfmarker] the effect of some of the echos. Um, a number of people have done this so that, say, if you're talking to a speakerphone, uh it makes it more like it would be, if you were talking right up to it. So this is sort of the st the straight forward approach. You say I [disfmarker] I [disfmarker] I want to use this uh [disfmarker] this item but I want to subtract off various kinds of echos. So you construct a filter, and you have this [disfmarker] this filtered version uh of the speech um gets uh uh [disfmarker] gets subtracted off from the original speech. Then you try to [disfmarker] you try to minimize the energy in some sense. And so um [disfmarker] uh with some constraints.
[speaker001:] Kind of a clean up thing, that [disfmarker]
[speaker005:] It's a clean up thing. Right.
[speaker001:] OK.
[speaker005:] So, echo cancelling is [disfmarker] is, you know, commonly done in telephony, and [disfmarker] and [disfmarker] and it's sort of the obvious thing to do in this situation if you [disfmarker] if, you know, you're gonna be talking some distance from a mike.
[speaker001:] When uh, I would have meetings with the folks in Cambridge when I was at BBN over the phone, they had a um [disfmarker] some kind of a special speaker phone and when they would first connect me, it would come on and we'd hear all this noise.
[speaker005:] Yeah.
[speaker001:] And then it was uh [disfmarker] And then it would come on and it was very clear, you know.
[speaker005:] Right. So it's taking samples, it's doing adaptation, it's adjusting weights, and then it's getting the sum. So um, uh anyway that's [disfmarker] that's kind of a reasonable thing that I'd like to have somebody try [disfmarker] somebody look [disfmarker] And [disfmarker] and the digits would be a reasonable thing to do that with. I think that'd be enough data [disfmarker] plenty of data to do that with, and i for that sort of task you wouldn't care whether it was uh large vocabulary speech or anything. Uh. [vocalsound] Um
[speaker002:] Is Brian Kingsbury's work related to that, or is it a different type of reverberation?
[speaker005:] Brian's [comment] Kingsbury's work is an example of what we did f f from the opposite dogma. Right? Which is what I was calling the "party line", which is that uh doing that sort of thing is not really what we want. We want something more flexible, uh i i where people might change their position, and there might be, you know [disfmarker] There's also um oh yeah, noise. So the echo cancellation does not really allow for noise. It's if you have a clean situation but you just have some delays, Then we'll figure out the right [disfmarker] the right set of weights for your taps for your filter in order to produce the effect of those [disfmarker] those echos. But um if there's noise, then the very signal that it's looking at is corrupted so that it's decision about what the right [disfmarker] you know, right [disfmarker] right uh [disfmarker] delays are [disfmarker] is, uh [disfmarker] is [disfmarker] right delayed signal is [disfmarker] is [disfmarker] is [disfmarker] uh is incorrect. And so, in a noisy situation, um, also in a [disfmarker] in a situation that's very reverberant [disfmarker] [comment] with long reverberation times [comment] and really long delays, it's [disfmarker] it's sort of typically impractical. So for those kind of reasons, and also a [disfmarker] a c a complete inversion, if you actually [disfmarker] I mentioned that it's kind of hard to really do the inversion of the room acoustics. Um, that's difficult because um often times the [disfmarker] the um [disfmarker] [vocalsound] the system transfer function is such that when it's inverted you get something that's unstable, and so, if you [disfmarker] you do your estimate of what the system is, and then you try to invert it, you get a filter that actually uh, you know, rings, and [disfmarker] and uh goes to infinity. So it's [disfmarker] so there's [disfmarker] there's [disfmarker] there's that sort of technical reason, and the fact that things move, and there's air currents [disfmarker] I mean there's all sorts of [disfmarker] all sorts of reasons why it's not really practical. So for all those kinds of reasons, uh we [disfmarker] we [disfmarker] we sort of um, concluded we didn't want to in do inversion, and we're even pretty skeptical of echo cancellation, which isn't really inversion, and um we decided to do this approach of taking [disfmarker] uh, just picking uh features, which were [disfmarker] uh will give you more [disfmarker] something that was more stable, in the presence of, or absence of, room reverberation, and that's what Brian was trying to do. So, um, let me just say a couple things that I was [disfmarker] I was gonna bring up. Uh. Let's see. I guess you [disfmarker] you actually already said this thing about the uh [disfmarker] about the consent forms, which was that we now don't have to [disfmarker] So this was the human subjects folks who said this, [comment] or that [disfmarker] that [disfmarker]?
[speaker002:] The a apparently [disfmarker] I mean, we're gonna do a revised form, of course. Um but once a person has signed it once, then that's valid for a certain number of meetings. She wanted me to actually estimate how many meetings and put that on the consent form. I told her that would be a little bit difficult to say. So I think from a s practical standpoint, maybe we could have them do it once every ten meetings, or something. It won't be that many people who do it [pause] that often, but um just, you know, so long as they don't forget that they've done it, I guess.
[speaker005:] OK. Um, back on the data thing, so there's this sort of one hour, ten hour, a hundred hour sort of thing that [disfmarker] that we have. We have [disfmarker] we have an hour uh that [disfmarker] that is transcribed, we have [disfmarker] we have twelve hours that's recorded but not transcribed, and at the rate we're going, uh by the end of the semester we'll have, I don't know, forty or fifty or something, if we [disfmarker] if this really uh [disfmarker] Well, do we have that much?
[speaker003:] Not really.
[speaker005:] Let's see, we have [disfmarker]
[speaker003:] It's three to four per week. So that's what [disfmarker] You know, that [disfmarker]
[speaker005:] uh eight weeks, uh is [disfmarker]
[speaker003:] So that's not a lot of hours.
[speaker005:] Eight weeks times three hours is twenty four, so that's [disfmarker] Yeah, so like thirty [disfmarker] thirty hours?
[speaker001:] Three [disfmarker] Three hours.
[speaker003:] Yeah. I mean, is there [disfmarker] I know this sounds [pause] tough but we've got the room set up. Um I was starting to think of some projects where you would use well, similar to what we talked about with uh energy detection on the close talking mikes. There are a number of interesting questions that you can ask about how interactions happen in a meeting, that don't require any transcription. So what are the patterns, the energy patterns over the meeting? And I'm really interested in this [vocalsound] but we don't have a whole lot of data. So I was thinking, you know, we've got the room set up and you can always think of, also for political reasons, if ICSI collected you know, two hundred hours, that looks different than forty hours, even if we don't transcribe it ourselves,
[speaker005:] But I don't think we're gonna stop at the end of this semester.
[speaker003:] so [disfmarker]
[speaker005:] Right? So, I th I think that if we are able to keep that up for a few months, we are gonna have more like a hundred hours.
[speaker003:] I mean, is there [disfmarker] Are there any other meetings here that we can record, especially meetings that have some kind of conflict in them [comment] or some kind of deci I mean, that are less well [disfmarker] I don't [disfmarker] uh, that have some more emotional aspects to them, or strong [disfmarker]
[speaker007:] We had some good ones earlier.
[speaker003:] There's laughter, um I'm talking more about strong differences of opinion meetings, maybe with manager types, or [disfmarker]
[speaker007:] I think it's hard to record those.
[speaker003:] To be allowed to record them?
[speaker005:] Yeah, people will get [disfmarker]
[speaker003:] OK.
[speaker002:] It's also likely that people will cancel out afterwards. But I [disfmarker] but I wanted to raise the KPFA idea.
[speaker003:] OK. Well, if there is, anyway.
[speaker005:] Yeah, I was gonna mention that.
[speaker007:] Oh, that's a good idea. That's [disfmarker] That would be a good match.
[speaker005:] Yeah. So [disfmarker] Yeah. So I [disfmarker] I [disfmarker] uh, I [disfmarker] I'd mentioned to Adam, and [disfmarker] that was another thing I was gonna talk [disfmarker] uh, mention to them before [disfmarker] [comment] that uh there's uh [disfmarker] It [disfmarker] it oc it occurred to me that we might be able to get some additional data by talking to uh acquaintances in local broadcast media. Because, you know, we had talked before about the problem about using found data, [comment] that [disfmarker] that uh it's just set up however they have it set up and we don't have any say about it and it's typically one microphone, in a, uh, uh [disfmarker] or [disfmarker] and [disfmarker] and so it doesn't really give us the [disfmarker] the [disfmarker] the uh characteristics we want. Um and so I do think we're gonna continue recording here and record what we can. But um, it did occur to me that we could go to friends in broadcast media and say "hey you have this panel show, [pause] or this [disfmarker] you know, this discussion show, and um can you record multi channel?" And uh they may be willing to record it uh with [disfmarker]
[speaker003:] With lapel mikes or something?
[speaker005:] Well, they probably already use lapel, but they might be able to have it [disfmarker] it wouldn't be that weird for them to have another mike that was somewhat distant.
[speaker003:] Right.
[speaker005:] It wouldn't be exactly this setup, but it would be that sort of thing, and what we were gonna get from UW, you know, assuming they [disfmarker] they [disfmarker] they start recording, isn't [disfmarker] als also is not going to be this exact setup.
[speaker003:] Right. No, I think that'd be great,
[speaker005:] So, [comment] I [disfmarker] I [disfmarker] I [disfmarker] I was thinking of looking into that.
[speaker003:] if we can get more data.
[speaker005:] the other thing that occurred to me after we had that discussion, in fact, is that it's even possible, since of course, many radio shows are not live, [comment] uh that we could invite them to have like some of their [disfmarker] [comment] record some of their shows here.
[speaker002:] Wow!
[speaker003:] Well [disfmarker] Or [disfmarker] The thing is, they're not as averse to wearing one of these head mount
[speaker007:] Right, as we are.
[speaker003:] I mean, they're on the radio, right? So. [comment] Um, I think that'd be fantastic
[speaker005:] Right.
[speaker003:] cuz those kinds of panels and [disfmarker] Those have interesting
[speaker005:] Yeah.
[speaker003:] Th that's an [disfmarker] a side of style [disfmarker] a style that we're not collecting here, so
[speaker005:] And [disfmarker] and the [disfmarker] I mean, the other side to it was the [disfmarker] what [disfmarker] which is where we were coming from [disfmarker] I'll [disfmarker] I'll talk to you more about it later [comment] is that [disfmarker] is that there's [disfmarker] there's uh
[speaker003:] it'd be great.
[speaker005:] the radio stations and television stations already have stuff worked out presumably, uh related to, you know, legal issues and [disfmarker] and permissions and all that. I mean, they already do what they do [disfmarker] do whatever they do. So it's [disfmarker] uh, it's [disfmarker] So it's [disfmarker] so it's another source. So I think it's something we should look into, you know, we'll collect what we collect here hopefully they will collect more at UW also and um [disfmarker] and maybe we have this other source. But yeah I think that it's not unreasonable to aim at getting, you know, significantly in excess of a hundred hours. I mean, that was sort of our goal. The thing was, I was hoping that we could [disfmarker] [@ @] in the [disfmarker] under this controlled situation we could at least collect, you know, thirty to fifty hours. And at the rate we're going we'll get pretty close to that I think this semester. And if we continue to collect some next semester, I think we should, uh [disfmarker]
[speaker003:] Right. Yeah I was mostly trying to think, OK, if you start a project, within say a month, you know, how much data do you have to work with. And you [disfmarker] you wanna s you wanna sort of fr freeze your [disfmarker] your data for awhile so um right now [disfmarker] and we don't have the transcripts back yet from IBM right?
[speaker005:] Well, we don't even have it for this f you know, forty five minutes, that was [disfmarker]
[speaker003:] Do [disfmarker] Oh, do we now? So um, not complaining, I was just trying to think, you know, what kinds of projects can you do now versus uh six months from now
[speaker005:] Yeah.
[speaker003:] and they're pretty different, because
[speaker007:] Right.
[speaker005:] Yeah. So I was thinking right now it's sort of this exploratory stuff where you [disfmarker] you look at the data, you use some primitive measures and get a feeling for what the scatter plots look like,
[speaker003:] um [disfmarker] Right.
[speaker005:] and [disfmarker] and [disfmarker] and uh [disfmarker] and meanwhile we collect, and it's more like yeah, three months from now, or six months from now you can [disfmarker] you can do a lot of other things.
[speaker003:] Right, right. Cuz I'm not actually sure, just logistically that I can spend [disfmarker] you know, I don't wanna charge the time that I have on the project too early, before there's enough data to make good use of the time. And that's [disfmarker] and especially with the student
[speaker007:] Right.
[speaker003:] uh for instance this guy who seems [disfmarker]
[speaker005:] Yeah.
[speaker003:] Uh anyway, I shouldn't say too much, but um if someone came that was great and wanted to do some real work and they have to end by the end of this school year in the spring, how much data will I have to work with, with that person. And so it's [disfmarker]
[speaker005:] i Yeah, so I would think, exploratory things now. Uh, three months from now [disfmarker] Um, I mean the transcriptions I think are a bit of an unknown cuz we haven't gotten those back yet as far as the timing, but I think as far as the collection, it doesn't seem to me l like, uh, unreasonable to say that uh in January, you know, ro roughly uh [disfmarker] which is roughly three months from now, we should have at least something like, you know, twenty five, thirty hours.
[speaker003:] And we just don't know about the transcription part of that,
[speaker005:] So that's [disfmarker]
[speaker002:] Yeah, we need to [disfmarker] I think that there's a possibility that the transcript will need to be adjusted afterwards,
[speaker003:] so. I mean, it [disfmarker]
[speaker002:] and uh es especially since these people won't be uh used to dealing with multi channel uh transcriptions.
[speaker003:] Right.
[speaker005:] Yeah.
[speaker002:] So I think that we'll need to adjust some [disfmarker] And also if we wanna add things like um, well, more refined coding of overlaps, then definitely I think we should count on having an extra pass through. I wanted to ask another a a aspect of the data collection. There'd be no reason why a person couldn't get together several uh, you know, friends, and come and argue about a topic if they wanted to, right?
[speaker005:] If they really have something they wanna talk about as opposed to something [@ @] [disfmarker] I mean, what we're trying to stay away from was artificial constructions, but I think if it's a real [disfmarker] Why not? Yeah.
[speaker003:] I mean, I'm thinking, politically [disfmarker]
[speaker007:] Stage some political debates.
[speaker002:] You could do this,
[speaker003:] Well yeah,
[speaker002:] you know. You could.
[speaker003:] or just if you're [disfmarker] if you ha If there are meetings here that happen that we can record even if we don't [pause] um have them do the digits, [comment] or maybe have them do a shorter [pause] digit thing [comment] like if it was, you know, uh, one string of digits, or something, they'd probably be willing to do.
[speaker007:] We don't have to do the digits at all if we don't want to.
[speaker003:] Then, having the data is very valuable, cuz I think it's um politically better for us to say we have this many hours of audio data, especially with the ITR, if we put in a proposal on it. It'll just look like ICSI's collected a lot more audio data. Um, whether it's transcribed or not um, is another issue, but there's [disfmarker] there are research questions you can answer without the transcriptions, or at least that you can start to answer.
[speaker007:] Yep.
[speaker002:] It seems like you could hold some meetings.
[speaker003:] So.
[speaker002:] You know, you and maybe Adam? You [disfmarker] you could [disfmarker] you could maybe hold some additional meetings, if you wanted.
[speaker001:] Would it help at all [disfmarker] I mean, we're already talking about sort of two levels of detail in meetings. One is uh um without doing the digits [disfmarker] Or, I guess the full blown one is where you do the digits, and everything, and then talk about doing it without digits, what if we had another level, just to collect data, which is without the headsets and we just did the table mounted stuff.
[speaker003:] Need the close talking mikes.
[speaker001:] You do, OK.
[speaker003:] I mean, absolutely,
[speaker005:] Yeah. Yeah.
[speaker003:] yeah. I'm really scared [disfmarker]
[speaker007:] It seems like it's a big part of this corpus is to have the close talking mikes.
[speaker003:] Um or at least, like, me personally? I would [disfmarker] [comment] I [disfmarker] couldn't use that data.
[speaker001:] I see, OK.
[speaker005:] Yeah.
[speaker002:] I agree.
[speaker003:] Um.
[speaker002:] And Mari also, we had [disfmarker] This came up when she she was here. That's important.
[speaker005:] Yeah,
[speaker003:] So it's a great idea,
[speaker005:] I [disfmarker] I [disfmarker] b By the [disfmarker] by the way, I don't think the transcriptions are actually, in the long run, such a big bottleneck.
[speaker003:] and if it were true than I would just do that, but it's not that bad [disfmarker] like the room is not the bottleneck, and we have enough time in the room, it's getting the people to come in and put on the [disfmarker] and get the setup going.
[speaker005:] I think the issue is just that we're [disfmarker] we're blazing that path. Right? And [disfmarker] and um [disfmarker] d Do you have any idea when [disfmarker] when uh the [disfmarker] you'll be able to send uh the ten hours to them?
[speaker007:] Well, I've been burning two C Ds a day, which is about all I can do with the time I have.
[speaker005:] Yeah. Yeah.
[speaker007:] So it'll be early next week.
[speaker005:] Yeah, OK. So early next week we send it to them, and then [disfmarker] then we check with them to see if they've got it and we [disfmarker] we start, you know asking about the timing for it.
[speaker007:] Yep.
[speaker005:] So I think once they get it sorted out about how they're gonna do it, which I think they're pretty well along on, cuz they were able to read the files and so on.
[speaker007:] Yep.
[speaker005:] Right?
[speaker007:] Yeah, but [disfmarker]
[speaker005:] Well [disfmarker]
[speaker007:] Yeah, who knows where they are.
[speaker001:] Have they ever responded to you?
[speaker007:] Nope.
[speaker005:] Yeah, but [disfmarker] You know, so they [disfmarker] they [disfmarker] they have [disfmarker] you know, they're volunteering their time and they have a lot of other things to do,
[speaker003:] What if [disfmarker]
[speaker007:] Yeah, you [disfmarker] we can't complain.
[speaker005:] right? But they [disfmarker] But at any rate, they'll [disfmarker] I [disfmarker] I think once they get that sorted out, they're [disfmarker] they're making cassettes there, then they're handing it to someone who they [disfmarker] who's [disfmarker] who is doing it, and uh I think it's not going to be [disfmarker] I don't think it's going to be that much more of a deal for them to do thirty hours then to do one hour, I think. It's not going to be thirty
[speaker007:] Yep. I think that's probably true.
[speaker003:] Really? So it's the amount of [disfmarker]
[speaker005:] It's [disfmarker] it's just getting it going.
[speaker007:] It's pipeline, pipeline issues.
[speaker003:] Right. What about these lunch meetings [disfmarker]
[speaker007:] Once the pipeline fills.
[speaker003:] I mean, I don't know, if there's any way without too much more overhead, even if we don't ship it right away to IBM even if we just collect it here for awhile, [comment] to record you know, two or three more meeting a week, just to have the data, even if they're um not doing the digits, but they do wear the headphones?
[speaker005:] But the lunch meetings are pretty much one person getting up and [disfmarker]
[speaker003:] No, I meant, um, sorry, the meetings where people eat their lunch downstairs, maybe they don't wanna be recorded, but [disfmarker]
[speaker007:] Oh, and we're just chatting?
[speaker003:] Just the ch the chatting.
[speaker007:] Yeah, we have a lot of those.
[speaker003:] I actually [disfmarker] I actually think that's [pause] useful [pause] data, um [pause] the chatting,
[speaker007:] Yeah, the problem with that is I would [disfmarker] I think I would feel a little constrained to [disfmarker] You know? Uh, some of the meetings [disfmarker]
[speaker003:] but [disfmarker] OK. You don't wanna do it, cuz [disfmarker] OK.
[speaker007:] You know, our "soccer ball" meeting? I guess none of you were there for our soccer ball meeting.
[speaker003:] Alright. Alright, [comment] so I'll just throw it out there,
[speaker007:] That was hilarious.
[speaker003:] if anyone knows of one more m or two more wee meetings per week that happen at ICSI, um that we could record, I think it would be worth it.
[speaker005:] Yeah. Well, we should also check with Mari again, because they [disfmarker] because they were really intending, you know, maybe just didn't happen, but they were really intending to be duplicating this in some level. So then that would double [pause] what we had. Uh. And there's a lot of different meetings at UW uh [disfmarker] I mean really m a lot more [comment] than we have here right cuz we're not right on campus,
[speaker007:] Right.
[speaker005:] so.
[speaker001:] Is the uh, notion of recording any of Chuck's meetings dead in the water, or is that still a possibility?
[speaker005:] Uh, [vocalsound] they seem to have some problems with it. We can [disfmarker] we can talk about that later. Um, but, again, Jerry is [disfmarker] Jerry's open [disfmarker] So I mean, we have two speech meetings, one uh network meeting, uh Jerry was open to it but I [disfmarker] I s One of the things that I think is a little [disfmarker] a little bit of a limitation, there is a think when the people are not involved uh in our work, we probably can't do it every week. You know? I [disfmarker] I [disfmarker] I [disfmarker] I think that [disfmarker] that people are gonna feel uh [disfmarker] are gonna feel a little bit constrained. Now, it might get a little better if we don't have them do the digits all the time.
[speaker007:] Yep.
[speaker005:] And the [disfmarker] then [disfmarker] so then they can just really sort of try to [disfmarker] put the mikes on and then just charge in and [disfmarker]
[speaker003:] What if we give people [disfmarker] you know, we cater a lunch in exchange for them having their meeting here or something?
[speaker002:] Well, you know, I [disfmarker] I do think eating while you're doing a meeting is going to be increasing the noise.
[speaker003:] OK. Alright, alright, alright.
[speaker002:] But I had another question, which is um, you know, in principle, w um, I know that you don't want artificial topics, but um it does seem to me that we might be able to get subjects from campus to come down and do something that wouldn't be too artificial. I mean, we could [disfmarker] political discussions, or [disfmarker] or something or other,
[speaker003:] No, definitely.
[speaker002:] and i you know, people who are [disfmarker] Because, you know, there's also this constraint. We d it's like, you know, the [disfmarker] the [disfmarker] uh goldibears [disfmarker] goldi goldilocks, it's like you don't want meetings that are too large, but you don't want meetings that are too small. And um [disfmarker] a and it just seems like maybe we could exploit the subj human subject p p pool, in the positive sense of the word.
[speaker001:] Well, even [disfmarker] I mean, coming down from campus is sort of a big thing, but what about
[speaker002:] We could pay subjects.
[speaker001:] or what about people in the [disfmarker] in the building?
[speaker003:] Yeah, I was thinking, there's all these other peo
[speaker001:] I mean, there's the State of California downstairs, and [disfmarker]
[speaker003:] Yeah. I mean [disfmarker]
[speaker007:] I just really doubt that uh any of the State of California meetings would be recordable and then releasable to the general public.
[speaker002:] Yeah.
[speaker001:] Oh.
[speaker003:] Mm hmm.
[speaker007:] So I [disfmarker] I mean I talked with some people at the Haas Business School who are i who are interested in speech recognition
[speaker003:] Alright, well.
[speaker007:] and, they sort of hummed and hawed and said "well maybe we could have meetings down here", but then I got email from them that said "no, we decided we're not really interested and we don't wanna come down and hold meetings." So, I think it's gonna be a problem to get people regularly.
[speaker001:] What about Joachim, maybe he can [disfmarker]
[speaker005:] But [disfmarker] but we c But I think, you know, we get some scattered things from this and that. And I [disfmarker] I d I do think that maybe we can get somewhere with the [disfmarker] with the radio. Uh i I have better contacts in radio than in television, but [disfmarker]
[speaker003:] Mm hmm.
[speaker001:] You could get a lot of lively discussions from those radio ones.
[speaker007:] Yep.
[speaker003:] Well, and they're already [disfmarker] they're [disfmarker] these things are already recorded,
[speaker005:] Yeah.
[speaker003:] we don't have to ask them to [disfmarker] even [disfmarker] and I'm not sure wh how they record it, but they must record from individual [disfmarker]
[speaker005:] n Well [disfmarker] No, I'm not talking about ones that are already recorded. I'm talking about new ones because [disfmarker] because [disfmarker] because we would be asking them to do something different.
[speaker003:] Why [disfmarker] why not? Well, we can find out. I know for instance Mark Liberman was interested uh in [disfmarker] in LDC getting [pause] data,
[speaker005:] Right, that's the found data idea.
[speaker003:] uh, and [disfmarker]
[speaker005:] But what I'm saying is uh if I talk to people that I know who do these th who produce these things we could ask them if they could record an extra channel, let's say, of a distant mike.
[speaker003:] Yeah. Mm hmm.
[speaker005:] And u I think routinely they would not do this. So, since I'm interested in the distant mike stuff, I wanna make sure that there is at least that somewhere
[speaker003:] Right. Great. OK.
[speaker005:] and uh [disfmarker] But if we ask them to do that they might be intrigued enough by the idea that they uh might be e e willing to [disfmarker] the [disfmarker] I might be able to talk them into it.
[speaker003:] Mm hmm.
[speaker007:] Um. We're getting towards the end of our disk space, so we should think about trying to wrap up here.
[speaker005:] OK. Well I don't [disfmarker] why don't we [disfmarker] why d u why don't we uh uh turn them [disfmarker] turn
[speaker003:] That's a good way to end a meeting.
[speaker007:] OK, leave [disfmarker] leave them on for a moment until I turn this off, cuz that's when it crashed last time.
[speaker002:] Oh. That's good to know.
[speaker005:] Turning off the microphone made it crash. Well [disfmarker]
[speaker002:] That's good to know.
[speaker005:] OK.