Attempting to solve a optical character recognition task using a feed-forward network
If you are serious about OCR systems and not just trying to learn CNN, start with a known good OSS product like tesseract. OCR some 1200 DPI grayscale documents, reverse engineer it to some degree so that you understand the design, join the development team, and pick up a ticket or two. Once you get into pull requests for a real system that is in constant use for a while, you will be an expert. Starting from scratch would make sense after you are an expert.