How repeatable are cognitive science findings within the same individual? Thanks to this website, I've seen a number of papers and scholarly articles that deal with cognition. I'm interested in how repeatable the findings are that are discovered as a result of experiments.

For example, when a mechanical system is tested, it is tested to work the same way every single time:


A lock opens and closes when a key is turned.
A car door opens up to the maximum degree allowed by design.


How much is a cognitive system like a mechanical system? Will it work the same way every time?

For example, I'm thinking about the design of an experiment I've been reading about: The Ikea Effect Experiment. Volunteers are asked to assemble Ikea boxes and bid on the box after assembly. The experiment shows that people value things they built themselves more than things that are just offered to them.

Now if the machine-like definition of repeatability is applied to this experiment: Would test subjects value boxes they built themselves next time they do an experiment? How about next month? What about in 3 years? Would people in the 1940's have the same response?

Have there been any studies that deal with this "repeatability" question? For example, studies that used the same test subjects doing the same experiment multiple times?
Humans are, as you would expect, much noisier than physical systems. As such, replication of experiments in psychology is very important. Replication provides evidence that the original results were not spurious.

As pointed out in the comments, replication with the same subject pool is not always the way to go. Many tasks have learning effects, and so you would not expect to achieve the same results by re-administering the experiment.

There is a vast statistical literature on reliability testing. A good place to start would be at the Wikipedia article for Reliability (psychometrics)