1. A computer-implemented method comprising: receiving a transcription of a first portion of a speech session, wherein the transcription of the first portion of the speech session is generated using a speaker adaptation profile; receiving a stability measure for a segment of the transcription; determining that the stability measure for the segment satisfies a threshold; in response to determining that the stability measure for the segment satisfies the threshold, triggering a real-time update of the speaker adaptation profile using the segment, or using a portion of speech data that corresponds to the segment; receiving a transcription of a second portion of the speech session, wherein the transcription of the second portion of the speech session is generated using the updated speaker adaptation profile; and outputting a set of transcriptions comprising the transcription of the first portion of the speech session and the transcription of the second portion of the speech session.