How to plot Loss Landscape with more than 2 weights in the network
It seems not possible to plot loss values (z) against all combinations of weights in all layers, especially when the network is big with thousands or millions of params; in that case, the number of points to plot is too too big.

And also, the 3D space can't be used to plot more than 3 dimensions.

However, with a deep network with lots of weights, these can be plotted:


Loss value against any pair of 2 weights
Turn the layer right before output layer (single neuron) into a layer of 2 neurons, and loss can be plotted against these 2 weights (but doesn't make much sense as the meaning of loss value depends on all other weights also)


Example plot when there are 2 neurons in the layer right before output layer (of 1 neuron):