Is there actually a lack of fundamental theory on deep learning? I heard several times that one of the fundamental/open problems of deep learning is the lack of "general theory" on it because actually we don't know why deep learning works so well. Even the Wikipedia page on deep learning has similar comments. Are such statements credible and representative of the state of the field?
There is a paper called Why does Deep Learning work so well?. 

"However, it is still not fully understood why deep learning works so well. In contrast to GOFAI (“good old-fashioned AI”) algorithms that are hand-crafted and fully understood analytically, many algorithms using artificial neural networks are understood only at a heuristic level, where we empirically know that certain training protocols employing large data sets will result in excellent performance. This is reminiscent of the situation with human brains: we know that if we train a child according to a certain curriculum, she will learn certain skills — but we lack a deep understanding of how her brain accomplishes this."