We investigated the neural systems that support number processing in a patient (JL) who had damage to the left ventral occipito-temporal cortex (LvOT). JL had severely impaired written word recognition but he was remarkably accurate in number tasks, albeit slower than normal. This suggests LvOT activation is necessary for efficient but not for accurate number decisions. Here we investigated how JL made accurate number decisions using fMRI; we compared JL's brain activation to that in healthy controls and in two patients with frontal lobe damage who, like JL, made slow but accurate responses in number tasks. For semantic relative to perceptual decisions on numbers, JL did not activate the left occipito-temporal area that was involved in all other subjects. However, JL had significantly increased activation in a left posterior middle temporal region. In addition, during semantic and perceptual decisions on numbers, JL showed increased activation in: (1) the right occipito-temporal cortex, (2) right caudate, and (3) bilateral frontal regions. These effects were unique to JL and cannot be explained in terms of abnormally long response times because they were not observed in the other patients who made slow but accurate number decisions. Together these results show that although the LvOT usually contributes to efficient number processing, activation in this region is not essential for accurate performance because (i) perceptual processing of numbers can be supported by right occipital, right caudate, and bilateral frontal activation and (ii) semantic processing of numbers can be supported by increased left posterior middle temporal activation associated with hand actions.