1. One or more computer-readable storage devices having computer-executable instructions, which when executed by one or more processing devices, cause the one or more processing devices to perform: receiving handwriting input including handwritten mathematical symbols; receiving speech input including spoken words corresponding to spoken mathematical symbols; recognizing the handwriting input to identify candidate handwritten mathematical symbols; recognizing the speech input to identify candidate spoken words; processing the candidate handwritten mathematical symbols into a handwriting graph including nodes representing the candidate handwritten mathematical symbols and arcs representing probabilities that the candidate handwritten mathematical symbols are correct; processing the candidate spoken words into a speech graph by: accessing a lookup table with the candidate spoken words recognized from the speech input to identify candidate spoken mathematical symbols; and including nodes in the speech graph representing the candidate spoken mathematical symbols and arcs representing probabilities that the candidate spoken mathematical symbols identified from the lookup table are correct; and enhancing the handwriting graph into an enhanced graph by adjusting individual probabilities in the handwriting graph based on individual matching probabilities in the speech graph.