1. A method of web crawling hidden files, comprising: retrieving a list of form controls from a web page; retrieving one or more candidate values for one of the form controls; retrieving an additional form control candidate value for one of the form controls, the additional form control candidate value not being shown in a static HTML description of the web page; generating form control values for the form controls based on the candidate values, the additional form control candidate value, and a knowledge base comprising characterizations of possible form control values, and N-grams generated from search query logs; submitting an event to the web page using the form control values; generating a URL for the form control values to crawl for the hidden files; and generating a plurality of URLs, the plurality of URLs being generated based on one or more constraints, the form control values being generated such that an estimated utility of the form control values is increased, the estimated utility being based on compliance with the constraints.