Which hyperparameters of a neural network can be tunned independently?
Your question is indeed a little bit broad, but I will try to explain to give you an overview on the importance and on some specific topics.

Indeed, multiple hyper-parameters exist in the context of deep learning. At the same time, just like Andrew Ng mentions in his courses, some are of an bigger importance than the others.

For instance, if you see that your training progresses very slowly (i.e. your convergence is relatively slow), you may want to fine-tune your learning rate.

Learning rate is a quintessential example of hyper-parameter that is more important than the number of neurons in a FullyConnected layer or to change the Dropout rate on a layer from 0.3 to 0.5.

At the same time, there exist two well known techniques for hyper-parameter search: grid search and random search. While the former behaves exactly like you explained (keeping the values of N-1 hyper-parameters fixed and iterating over some specific values of the N hyper-parameter), the random search has proven to have a better positive impact, as it slightly modifies all your hyper-parameters after a search step; although it may not be intuitive at first sight, this can yield better and earlier results than the grid search.