1. A computer-implemented process for computing excluded data, the computer-implemented process comprising: identifying a web page of interest to form an identified page; loading the identified page a first time to form a first load; responsive to a determination that a delta has not been computed for the identified web page, loading the identified page a second time to form a second load, wherein the second load is based, at least in part, upon the use of a proxy; determining whether portions of the first load differ from portions of the second load; responsive to a determination portions of the first load differ from portions of the second load, identifying the portions that differ to form a delta; storing the delta to form a stored delta; excluding the stored delta from a document object model associated with the identified page to form a modified document object model; excluding the stored delta from a document object model comparison process, wherein the document object model comparison process from which the stored delta is excluded is a document object model equivalence function, wherein the excluded stored delta includes one or more page sections ignored by crawlers; and if the identified page is part of a rich Internet application, adding the identified page to a rich Internet application model.
1. A computer-implemented process for computing excluded data, the computer-implemented process comprising: identifying a web page of interest to form an identified page; loading the identified page a first time to form a first load; responsive to a determination that a delta has not been computed for the identified web page, loading the identified page a second time to form a second load, wherein the second load is based, at least in part, upon the use of a proxy; determining whether portions of the first load differ from portions of the second load; responsive to a determination portions of the first load differ from portions of the second load, identifying the portions that differ to form a delta; storing the delta to form a stored delta; excluding the stored delta from a document object model associated with the identified page to form a modified document object model; excluding the stored delta from a document object model comparison process, wherein the document object model comparison process from which the stored delta is excluded is a document object model equivalence function, wherein the excluded stored delta includes one or more page sections ignored by crawlers; and if the identified page is part of a rich Internet application, adding the identified page to a rich Internet application model. 4. The computer-implemented process of claim 1 wherein identifying the portions that differ to form a delta comprises: identifying the portions that differ as differences between a document object model of the first load and a document object model the second load.