Why object detection algorithms are poor in optical character recognition?
Good question! Using Yolo to recognise characters would be a good experiment to try. It may be because of the density of characters on a page -- systems like Yolo are very good at detecting a small number e.g. 2,3 or 10, objects, but don't work so well when the number of objects  is the hundreds as you might have with OCR. A better approach might be to try face detection methods that work well with large crowds.