Should I Plot my Learning Curve Depending on Various Parameters to Diagnose Overfitting/Underfitting or is One Parameter Sufficient? I know that learning curves are a good tool to diagnose overfitting or underfitting for a model. 

Their working principle is simple: The training/validation loss/accuracy is plotted depending on specific parameters, e.g., the learning rate, the model architecture complexity, the training set size, the number of training epochs, etc.

However, should I always plot many different learning curves, i.e., depending on these different parameters? Or is it sufficient, if I only plot one learning curve depending on just one parameter? Thus, would it be reliable enough, if I would just plot the learning curve based on the keras (tensorflow 2 as the backend) model.fit history, i.e., the training/validation loss depending on the training epochs.
Overfitting is a result of the entire model. It is best to visualize every part of the model to understand how the combination of elements is overfitting.

Ideally, you should visualize all elements on the same interactive figure so how the combination is overfitting. If that is not possible or the interpretation is difficult, then many single element visualizations would be an option. However, that increases the possibility of missing interactions causing overfitting.