What experiment in a simple decision task should we run to obtain one million trials? Context: Models of cognitive processes require very large datasets to be fitted. Sadly, it is difficult for a single laboratory to achieve this alone. I propose that we work collectively and collaboratively in realizing a 1 million RT experiment. This initiative is described on my ResearchGate page, www.researchgate.net/profile/Denis_Cousineau. 

Beyond model fitting, such a dataset will be beneficial for other purposes. Examples: Are parameters describing performances normally distributed across subjects? Does training effect --resulting in faster RT-- imply that the parameters are all evolving in one direction? Does Bayesian estimation more apt than Maximum Likelihood at characterizing the parameters? How can we estimate non-decision times? What is the shape of the RT distributions and how training effect alter its shape? In Bayes again, after how many trials does priors have no longer an impact on the estimates? etc.

Task Requirements:
The task must be behavioral only (response times and response choices) as it is not possible to move EEG or fMRI material all around the world. The precise task used is not a critical constraint because the current models of cognition (sampling models such as LBA or Diffusion) are very flexible; they aim at predicting performance in a wide variety of situations.  

The tasks proposed will be evaluated based on the following criteria criteria: 


small number of response alternatives
few conditions
low error rates
stimuli that can be characterized for modeling purposes
possibility of extended training on that task. 


Possible candidate tasks could include: A redundant target detection task (Miller, 1982) or a same-different task (e.g., Bamber, 1969). These tasks are simple and accuracy is typically high (above 90% correct); they are possibly performed using some form of parallel processing; response times are generally well below 700 ms.

So in summary:

What task and experimental design would be most suitable given the above constraints?

Please let's see what are the possible tasks, comment on others' proposals, and if you find the perfect task, upvote it. Have your colleagues make comments or propose new tasks.
I like this idea! I think the task should be one where the data could be used by many analytic tools. Hence, the data would not only be beneficial for one particular question, for example parameter estimations, but also for other tools such as Systems Factorial Technology (see Townsend &amp; Nozawa, 1995), which provides a deeper understanding of the underlying cognitive sub-processes architecture, and other interesting information such as the process' capacity.

I suggest that the task at least respect a double factorial paradigm. This design is very simple: you have four conditions in a 2 x 2 design, where you vary two variables. It could be any variable, the only restriction is that you have to vary them independently one from another. In other words, each variable should reflect a cognitive sub-process. In one condition, both variables are in scenario where the sub-process should work very effectively (or optimally). In an other condition, both variables are in a scenario where the sub-process should work less effectively (or sub-optimally). And in the two remaining conditions, one variable works optimally and the other sub-optimally, and vice-versa.

As for the very details of the task, such as the stimuli that will be used and the specific variables, I do not have preferences. Letters are very manipulable, but they do carry semantic (more or less). Abstract stimuli could work. We could play with color, shapes, complexity, etc. At Psychonomics this year, they were many experiments which have use this design.

Reference for citation:
Townsend, J. T. &amp; Nozawa, G. (1995). Spatio-temporal properties of elementation perception an investigation of parallel, serial, and coactive theories. Journal of Mathematical Psychology, 39(4), 321–359. doi: 10.1006/jmps.1995.1033