But this is what people do in a sense. After repeated experience people tend to pay less attention to the actually experience the senses are having and start to work from memory. Buddhism and various traditions talk against this way of act strictly from memory.