1. A computer-implemented method for query processing, comprising: under the control of one or more computer systems configured with executable instructions, receiving user input data; storing the input data in one or more storage nodes by at least: storing a first portion of the user input data in an append data store as a result of having insufficient computing capacity to perform one or more optimization operations; extracting a second portion of the user input data from the append data store and performing the one or more optimization operations on the second portion of the input data to create optimized data as a result of regaining sufficient computing capacity to perform the one or more optimization operations; and storing the optimized data in an optimized data store; receiving a user query and a completion threshold based at least in part on information generated by the user; performing a search for one or more records responsive to the user query, using a filter, on the optimized data store; obtaining sufficient records to satisfy the completion threshold by at least: if the search on the optimized data store obtained sufficient records to satisfy the completion threshold, returning a result of the search in response to the query; and if the search on the optimized data store did not obtain sufficient records to satisfy the completion threshold, performing a second search for the one or more records response to the user query, using the filter, on the append data store; and providing the one or more records.