5. A system of classifying marking types on images of a document, the system comprising: a segmenter operated on a processor and configured to receive the document containing the images, the segmenter segmenting the images into fragments of foreground pixel structures that are identified as being likely to be of the same marking type by finding connected components, and dividing at least some of the connected components to obtain image fragments, the connected components being isolated, continuous regions of foreground pixels, the segmenter segmenting the images by: identifying neatly written or printed text by grouping selected feature points along predetermined orientations, the feature points being local extrema of bounding contours of the connected components; and subtracting enclosing boundary boxes of text lines from remaining document material to fragment connected components that are partly part of the text lines and partly part of extraneous markings; and a classifier operated on a processor and configured to receive the fragments, the classifier providing a category score to each received fragment, wherein the classifier is trained from ground truth images whose pixels are labeled according to known marking types, the classifier assigning a same label to all pixels in a fragment when the fragment is classified by the classifier.
15. The system according to claim 5 , wherein the segmenter segments the images by further: determining the enclosing boundary boxes for individual text lines based on the grouping.