Neural network for PDE: Should we train the PDE using more initial and boundary data at the beginning?
There is sadly very little theory that is useful for predicting the behavior of neural networks.  Also, their effectiveness tends to be dependent on the particular workload/task you are trying to solve it.  While I know it's not what you were hoping to hear, I suggest that you try it in empirical experiments.