15. A method of replacing one or more words in a transcription generated by a remote automatic speech recognition (ASR) engine, wherein the method is performed by a portable computing system having a processor and a memory, the method comprising: maintaining a personal vocabulary that includes replacement words; wherein the replacement words in the personal vocabulary are obtained from personal data associated with a user; and wherein the personal data is obtained from: stored contact data for the user, stored calendar data for the user, text-based messages sent or received by the user; or a social network of which the user is a member, receiving a transcription of an audio recording, wherein the transcription is generated by the remote ASR engine using an ASR vocabulary, wherein the transcription includes a transcribed word that represents a spoken word in the audio recording, and wherein the remote ASR engine is located geographically remotely from the portable computing system; receiving data associated with the transcribed word, wherein the data associated with the transcribed word includes a word lattice and associated confidence scores, and wherein the confidence scores are generated by the remote ASR engine; receiving audio data that includes the spoken word; generating a second transcription using the audio data, wherein the second transcription is generated by a local ASR engine using the personal vocabulary and re-scores the word lattice; identifying a replacement word from the second transcription; and creating a rule to automatically replace the transcribed word with the replacement word whenever the transcribed word is found in a transcription, and transmitting the rule or the re-scored word lattice to the remote ASR engine from which a vocabulary of the remote ASR engine is modified; wherein the portable computing system is a mobile phone or table, and wherein the replacement word is from the personal vocabulary.
23. The method of claim 15 , wherein the threshold confidence level depends at least in part on a weighting associated with the replacement word.