1. A method of indexing data and merging indexes within a database management system, the method comprising the following steps: assign a first temporal indicator associated with a start time of a first crawl to a first instance of an index generated by the first crawl, wherein the first instance of the index comprises a first index of a first plurality of indexes generated by the first crawl and wherein the first plurality of indexes are database files; assign the first instance of the index to a set of instances of the index to be merged; assign a second temporal indicator associated with a start time of a second crawl to a second instance of an index generated by the second crawl, wherein the second instance of the index comprises a second index of a second plurality of indexes generated by the second crawl and wherein the second plurality of indexes are database files; assign the second instance of the index to the set of instances to be merged; store the set of instances of the index to be merged; validate the set of instances to be merged so as to generate valid instances; merge the valid instances to create a merged index instance; and generate an index update dictionary that includes a table comprising meta-data about the instances of the index, an identifier for the table, a state indictor including the states StateNew when an object of the index is created but is not yet used, StateOpen when the index is in use but does not yet include any committed data, StateClosed when the index instance includes committed data and processing is complete and StatelnMerge when the index instance is being merged with other indexes, and an element that includes a number of rows in the index and an element that includes an estimate of a size of the instances of the index.