Number segmentation techniques
As with all ocr, you need to segment each individual character. Assuming this step is already done (and maybe some post processing to One easy way is to take the number and reduce the resolution. Similar to a lcd character display. 

You can then do a simple correlation between the reduced resolution number from the image, and a template. Since the numbers on street signs are printed and very consistent a simple template match could work well.

your template should contain the numbers 0-9 in a font that closely matches those used on your road signs.