Methods to prevent overfitting
When choosing a specific model, to which you will try to fit your data into, you must know something about the problem. There is no way around it, machine learning cant do this magic for us, we have to know something. 

First, convince yourself why complicated models can cause overfitting. Suppose I'm trying to learn some real valued function $f$, and I'm given $n$ samples of the form $S=(x_1,y_1),...,(x_n,y_n)$. If i choose to model my data by some polynomial (of arbitrary degree) then for any set of samples $S$, i can find a function in my model with zero training error, although it is likely to have high generalization error (a degree $n$ polynomial passing through all points in $S$ can always be found, regardless of any structure of the points in $S$). Perhaps using polynomials of degree$ \le 3$ will be good enough to model my data (it doesn't suffer from the previous problem). Why 3? I don't know, this depends on the specific types of functions I'm trying to learn.

A more concrete example, suppose you want to learn what is the appropriate dosage of some medicine, and you are given samples of the form $(x_1,y_1),...,(x_n,y_n)$ where $x_i\in\mathbb{R}$ denotes the dosage, and $y_i\in\left\{\text{'good'},\text{'bad'}\right\}$ tells you whether the dosage $x_i$ worked (it can be too high or too low). Allowing your algorithm to choose some arbitrary function can result in strange outputs. For example if you received the samples $(1,\text{'bad'}),(2,\text{'good'}),(3,\text{'good'})$ then a function $f:\mathbb{R}\rightarrow\left\{-1,1\right\}$ which satisfies $f(1)=-1,f(2)=f(3)=1$, but is random everywhere else, is consistent with the samples. Such $f$ will probably have no indication on the true effect of the medicine. Now, I'm no doctor, but perhaps it is worth considering functions of the form:

$I_{a,b}=\begin{cases}
+1, &amp; x\in [a,b] \\
-1, &amp; \text{otherwise}
\end{cases}$. 

When choosing to model my data using such functions (segments), i use my knowledge on how medicines work (there is some appropriate dosage interval, while large deviations to any side could be dangerous).