doubt in improvement of performances given by a layer in a deep neural network Today I was discussing about neural network with a friend, and he said that the more layers we add, the less increase in accuracy each layer gives individually. Is this fact true? 

I know that it is better to go deep then wide when building a neural network, but it is true that if we have a lot of layers, individually they give a smaller contribution to the performances? Thanks in advance.
Yes, there is a threshold where additional layers are too much.
Look at  residual networks