How to scale up algorithm development?
First off, if your data has as many variations (in function of time, context, and others) as to make it hard to apply a single strategy to cope with it, you may be interested in doing a prior temporal/contextual/... characterization of the dataset. Characterizing data, i.e., extracting information about how the volume or specifics of the content varies according to some criteria, usually provides with a better understanding (more consise and precise) than simply inferring algorithms on a brute-force fashion.

So, answering each question:


characterization is definitely a means of reducing the number of iterations while trying to select proper algorithms for specific data;
if you have a discrete set of criterias on which your data varies, it becomes much easier to scale up solutions, as will know what information you'd gain/lose if simpler/specific solutions were applied;
after a characterization, you should be also easier to select parameters, since you'd know what kind of specific data you'd be dealing with;
finally, you may use data mining/machine learning algorithms to support this characterization. This includes using:

clustering algorithms, to reduce the dimensionality of data;
classification algorithms, to help deciding on specific properties the data in function of time/context/... may present;
association rules, to predict particular knowledge from the dataset, while also improving/fine-graining the data used for later analysis;
and other possible strategies and analyses.



And here is a list of some criterias on which to analyse data, which you may find helpful.