Methods to prevent overfitting I am aware of three approaches to prevent over-fitting of data when trying to model it on a neural net.

The first two approaches I know suggest to train on more data and employ bootstrap aggregating. The third approach is to use a model that has the right capacity, one that has enough to fit the true regularities but not the weaker/dubious regularities (the noise). However, I don't quite understand this 2nd approach. How do we create a distinction and choose a model that detects this? Can someone expand on the second approach? I am aware of ways to limit the capacity such as early stopping(which is self explanatory), penalty for peculiar weights, etc. But an explanation on why these specific methods limit the capacity to fit the right data but not the wrong one would be helpful.
When choosing a specific model, to which you will try to fit your data into, you must know something about the problem. There is no way around it, machine learning cant do this magic for us, we have to know something. 

First, convince yourself why complicated models can cause overfitting. Suppose I'm trying to learn some real valued function $f$, and I'm given $n$ samples of the form $S=(x_1,y_1),...,(x_n,y_n)$. If i choose to model my data by some polynomial (of arbitrary degree) then for any set of samples $S$, i can find a function in my model with zero training error, although it is likely to have high generalization error (a degree $n$ polynomial passing through all points in $S$ can always be found, regardless of any structure of the points in $S$). Perhaps using polynomials of degree$ \le 3$ will be good enough to model my data (it doesn't suffer from the previous problem). Why 3? I don't know, this depends on the specific types of functions I'm trying to learn.

A more concrete example, suppose you want to learn what is the appropriate dosage of some medicine, and you are given samples of the form $(x_1,y_1),...,(x_n,y_n)$ where $x_i\in\mathbb{R}$ denotes the dosage, and $y_i\in\left\{\text{'good'},\text{'bad'}\right\}$ tells you whether the dosage $x_i$ worked (it can be too high or too low). Allowing your algorithm to choose some arbitrary function can result in strange outputs. For example if you received the samples $(1,\text{'bad'}),(2,\text{'good'}),(3,\text{'good'})$ then a function $f:\mathbb{R}\rightarrow\left\{-1,1\right\}$ which satisfies $f(1)=-1,f(2)=f(3)=1$, but is random everywhere else, is consistent with the samples. Such $f$ will probably have no indication on the true effect of the medicine. Now, I'm no doctor, but perhaps it is worth considering functions of the form:

$I_{a,b}=\begin{cases}
+1, &amp; x\in [a,b] \\
-1, &amp; \text{otherwise}
\end{cases}$. 

When choosing to model my data using such functions (segments), i use my knowledge on how medicines work (there is some appropriate dosage interval, while large deviations to any side could be dangerous).