1. A system comprising: a parsing block for pre-processing of a document indexing application; a filtering block for discarding irrelevant documents; a transformation block to clean up and annotate input data by identifying at least one document language; a processor configured for grouping inputs to collect documents for a same entity in a single spot, the processor processing a query operation; a data package including a data result component, the data result component including data generated based on successful completion of at least a portion of the query operation; wherein the data package also includes an error information data component based on one or more errors encountered during processing of the query operation; the processor continuing, as a function of at least one of a number of errors encountered and a class of error encountered, the query operation on data results relating to portions of the query operation that successfully completed; an indicator to separate and distinguish the error information data from the data result; a memory associated with a distributed file system for storing a final write output relating to the query operation; and a compiler that uses the data package for generating a program to optimize processing time by establishing a declarative specification of error tolerance based on previous results.