1. A computer-implemented method, comprising: accessing a plurality of parameters for a speech recognition module having a front-end feature extraction module that produces feature vectors from input speech that are provided to a decoder of the speech recognition module that utilizes a back-end acoustic model having acoustic model parameters to generate an output for the input speech, the plurality of parameters including feature extraction parameters used by the front-end feature extraction module and acoustic model parameters used by the back-end acoustic model; accessing a plurality of training audio signals and corresponding text transcriptions for the training audio signals; applying the plurality of training audio signals to the speech recognition module using the plurality of parameters and determining an output of the speech recognition module based on the training audio signals; comparing the output of the speech recognition module and the corresponding text transcriptions; applying an objective function to the speech recognition module, based on the comparison of the output of the speech recognition module and the corresponding text transcriptions, to determine an objective function value based on the plurality of parameters, wherein the objective function value is determined based on correct outputs for the training audio signals, and wherein applying the objective function further includes determining a gradient of the objective function with respect to each parameter of the acoustic model parameters and determining a gradient of the objective function with respect to each parameter of the feature extraction parameters; using the gradients and the objective function value to jointly train the feature extraction parameters and the acoustic model parameters by adjusting at least one of the feature extraction parameters and at least one of the acoustic model parameters to increase the objective function value using a processor of the computer, wherein the front-end feature extraction module is trained using the objective function value and the gradients determined with respect to each parameter of the feature extraction parameters and the back-end acoustic model is trained using the objective function value and the gradients determined with respect to each parameter of the acoustic model parameters; and providing an adjusted speech recognition module as a function of the adjustment of the at least one of the plurality of parameters.