16. The computer-implemented method of claim 1 , wherein the base rules further comprise function rules, variable rules, and dynamic rules.
1. A computer-implemented method for discovering links in a script, the method comprising: receiving webpages associated with one or more scripts, wherein the one or more scripts are Javascripts; processing the webpages to locate the one or more Javascripts, wherein processing the webpages to locate the Javascripts further comprises: extracting markup language tags from the webpages to locate function calls, variables, and constants; and identifying script elements and non-script elements based on the markup language tags; accessing rules corresponding to the one or more Javascripts; parsing the one or more Javascripts based on the rules corresponding to the one or more Javascripts, wherein the rules include base rules that are applied to all web pages, site rules that are applied to all webpages from a specific site, and auto-discovered rules that are applied to specific webpages; identifying segments of the one or more Javascripts that satisfy the rules; evaluating the identified segments of the one or more Javascripts and applying the rules to the extracted function calls, variables, and constants to generate links; storing the generated links in an index; and retrieving content associated with the generated links; optimizing the retrieved content; and storing the optimized content and metadata with the generated links, wherein the metadata comprises types of content associated with the generated links, types of files associated with the generated links, dialog attributes associated with the generated links, pop-up attributes associated with the generated links, and display sizes of the content associated with the generated links.