Does associating a color with numbers improve math learning? I have heard a number of reports that some mathematical savants associate particular colors with numbers.  It got me wondering, if colors are associated with numbers during mathematical teaching, would this improve ability?

For example, if every time I showed my child the number 2, it was colored red, and every time I showed her 3, it was blue, and so on, would she learn arithmatical skills like addition, subtraction, multiplication, and division faster?

Further, would it matter which colors were associated with which numbers?  Should 5 be purple?
I don't know of research that answers this question directly, but I'm going to guess the answer is no, it wouldn't help, based on the following reasoning.
First, people tend to learn math less well when superfluous visual richness is added. I think adding color to numbers counts as superfluous visual richness.

Brown, M. C., McNeil, N. M., &amp; Glenberg, A. M. (2009). Using concreteness in education: Real problems, potential solutions. Child Development Perspectives, 3(3), 160–164.
Mayer, R. E., Sims, V., &amp; Tajika, H. (1995). A Comparison of How Textbooks Teach Mathematical Problem Solving in Japan and the United States. American Educational Research Journal, 32(2), 443–460.

Second, in general, associating something you want people to recall with a larger number of cues is going to help recall.  (Sorry I don't know a citation for this off the top of my head, but I think it's a general principle of information theory.)  In this case, the colors and the numerals are both cues for numbers, which should lead to better recall than if you only had the numerals.  However, you are looking at learning arithmetic and not learning the number sequence.
Learning numbers MIGHT go faster with this additional cue. However, I'd be concerned whether there might be some negative impact on ability to use numerals once the color cue is removed, as it inevitably will be.
Presenting numbers together with analogue representations of magnitude does aid in learning arithmetic facts:

Booth, J. L., &amp; Siegler, R. S. (2008). Numerical magnitude representations influence arithmetic learning. Child Development, 79(4), 1016–1031.

However, analogue representations of magnitude are a cue that's relevant to the actual meaning of the numbers, while colors are not. I doubt colors would deliver this particular benefit.
With all that said, I haven't considered motivational issues.  If children are motivated by brightly colored number blocks, then I'd say go ahead and use them.  I doubt that the cognitive effects would be strong one way or the other, so even if negative, they'd probably be outweighed by any motivational benefits.