14. A computer-readable storage device with instructions stored thereon that when executed by one or more processing units, cause the one or more processing units to perform a method for speech recognition with local and remote feedback loops, the method comprising: collecting user data including audio and textual data, wherein the textual data is from user generated documents; receiving directed feedback from the user in response to recognized speech, the feedback comprising at least one of audio or textual data; aggregating the collected user data and the directed feedback; filtering the aggregated data at different levels for the local and the remote feedback loops to protect private data; storing the data filtered at the local feedback level to customize current language and acoustic models; providing the data filtered at the remote feedback level to a system developer to enable the system developer to update speech recognition generic language and acoustic models; receiving speech inputs; and recognizing, by a speech recognition system, the speech inputs based at least in part on the customized current language and acoustic models and the updated generic language and acoustic models.
14. A computer-readable storage device with instructions stored thereon that when executed by one or more processing units, cause the one or more processing units to perform a method for speech recognition with local and remote feedback loops, the method comprising: collecting user data including audio and textual data, wherein the textual data is from user generated documents; receiving directed feedback from the user in response to recognized speech, the feedback comprising at least one of audio or textual data; aggregating the collected user data and the directed feedback; filtering the aggregated data at different levels for the local and the remote feedback loops to protect private data; storing the data filtered at the local feedback level to customize current language and acoustic models; providing the data filtered at the remote feedback level to a system developer to enable the system developer to update speech recognition generic language and acoustic models; receiving speech inputs; and recognizing, by a speech recognition system, the speech inputs based at least in part on the customized current language and acoustic models and the updated generic language and acoustic models. 17. The computer-readable storage device of claim 14 , wherein the filtered data is provided to the system developer through one of: an email exchange, a file transfer, or an access through a hosted service.