12. A computing system having processor and hardware memories for discovering links in a script of a webpage, the system comprising: a crawler accessing web pages to identify scripts associated with the webpages, wherein the webpages are HTML pages; a rules engine that parses the identified scripts and evaluates portions of the identified scripts based on rules that detect link-generating segments of the identified scripts, wherein the segments of the identified scripts are evaluated based on variables and expressions specified in the identified scripts and matching function patterns located in the segment and the rules to detect links generated by the identified scripts; an index to store the detected links and metadata for the detected links, wherein the metadata comprises types of content associated with detected links, types of files associated with the detected links, dialog attributes associated with the detected links, pop-up attributes associated with the detected links, and display sizes of the content associated with the detected links; and the processor that retrieves the content associated with the detected links; optimizes the retrieved content; and optimized content in the index with the detected links.
12. A computing system having processor and hardware memories for discovering links in a script of a webpage, the system comprising: a crawler accessing web pages to identify scripts associated with the webpages, wherein the webpages are HTML pages; a rules engine that parses the identified scripts and evaluates portions of the identified scripts based on rules that detect link-generating segments of the identified scripts, wherein the segments of the identified scripts are evaluated based on variables and expressions specified in the identified scripts and matching function patterns located in the segment and the rules to detect links generated by the identified scripts; an index to store the detected links and metadata for the detected links, wherein the metadata comprises types of content associated with detected links, types of files associated with the detected links, dialog attributes associated with the detected links, pop-up attributes associated with the detected links, and display sizes of the content associated with the detected links; and the processor that retrieves the content associated with the detected links; optimizes the retrieved content; and optimized content in the index with the detected links. 15. The computing system of claim 12 , wherein the rules check functions, variables, and event handlers specified in the identified scripts and a subset of the rules are scoped to correspond to a subset of the webpages.