Why are linear phase filters called so, if they provide Constant delay instead of linear Linear phase filters delay all frequencies by the same amount. Why aren't they called Constant phase filters instead of Linear phase? 

As I understand, if there is an input signal with two components $f_1$ and $f_2$, and it is passed through a linear phase system which introduces a delay $t$, $f_1$ is delayed by $t$, and $f_2$ is also delayed by $t$. 

To what property does the phase have a linear relationship with? Could someone provide some insight into this? Some Math would be helpful.
Your premise is wrong. Linear phase filters offer linear phase.

Phase is not the same as time delay – at $f_1$, time delay $t$ will lead to a phase of $\varphi_1=-\omega_1\cdot t$, whereas at $f_2$, the phase will be $\varphi_2=-\omega_2\cdot t$ ($\omega$ is $2\pi f$); as you can immediately see, phase is a linear function of frequency.

(It's always good to remind oneself that frequency is just the derivative of phase over time, $\omega=\frac{d\varphi}{dt}$, and thus phase is just an integral of frequency over time.)