Should deep residual networks be viewed as an ensemble of networks?
Imagine a genie grants you three wishes. Because you are an ambitious deep learning researcher your first wish is a perfect solution for a 1000-layer NN for Image Net, which promptly appears on your laptop.

Now a genie induced solution doesn't give you any intuition how it might be interpreted as an ensemble, but do you really believe that you need 1000 layers of abstraction to distinguish a cat from a dog? As the authors of the "ensemble paper" mention themselves, this is definitely not true for biological systems.

Of course you could waste your second wish on a decomposition of the solution into an ensemble of networks, and I'm pretty sure the genie would be able to oblige. The reason being that part of the power of a deep network will always come from the ensemble effect.

So it is not surprising that two very successful tricks to train deep networks, dropout and residual networks, have an immediate interpretation as implicit ensemble. Therefore "it's not depth, but the ensemble" strikes me as a false dichotomy. You would really only say that if you honestly believed that you need hundreds or thousands of levels of abstraction to classify images with human accuracy. 

I suggest you use the last wish for something else, maybe a pinacolada.