15. An apparatus for computing excluded data, the apparatus comprising: a communications fabric; a memory connected to the communications fabric, wherein the memory contains computer executable program code; a communications unit connected to the communications fabric; an input/output unit connected to the communications fabric; a display connected to the communications fabric; and a processor unit connected to the communications fabric, wherein the processor unit executes the computer executable program code to direct the apparatus to: identify a web page of interest to form an identified page; load the identified page a first time to form a first load; responsive to a determination that a delta has not been computed for the identified web page, load the identified page a second time to form a second load; wherein the second load is based, at least in part, upon the use of a proxy; determine whether portions of the first load differ from portions of the second load; responsive to a determination portions of the first load differ from portions of the second load, identify the portions that differ to form a delta; store the delta to form a stored delta; exclude the stored delta from a document object model associated with the identified page to form a modified document object model; exclude the stored delta from a document object model comparison process, wherein the document object model comparison process from which the stored delta is excluded is a document object model equivalence function, wherein the excluded stored delta includes one or more page sections ignored by crawlers; and if the identified page is part of a rich Internet application, add the identified page to a rich Internet application model.
18. The apparatus of claim 15 wherein the processor unit executes the computer executable program code to identify the portions that differ to form a delta further directs the apparatus to: identify the portions that differ as differences between a document object model of the first load and a document object model the second load.