How is cross-correlation related with orthogonality? In linear prediction we can say that in case of optimum linear predictor the error with be orthogonal to data. And when we derive minimum mean square error for $\underline{y} = \mathbf{a}\underline{x} + \underline{b}$ case we find following relation


  $\xi = \sigma^2_y (1 - \rho_{xy} )^2$


Where $\rho_{xy} $ is correlation coefficient and if we talk about cross correlation $r_{xy}(k,l)$ of zero mean and uncorrelated random variables x and y, then how does this conclude that if $r_{xy}(k,l)$ will be then the signals will be orthogonal?
I suppose you mean the cross-correlation at lag zero. Well take an Hilbert space $H$ (i.e. a metric space in which you can define a scalar product $\langle\cdot ,\cdot\rangle$). Then $x,y\in H$ are orthogonal if $\langle x,y\rangle=0$, by definition.
If your Hilbert Space is $L_2(\mathbb{R})$ (the space of real square integrable functions) then the scalar product of $f,g\in L_2(\mathbb{R})$ is defined as
$$
\langle f, g\rangle = \int_{-\infty}^\infty \overline{f(t)}g(t) dt
$$
If the signals are assumed to be ergodic, which is actually a costumary assumption, then the cross-correlation is simply a sliding scalar product, you have
$$
\text{corr}(f,g)(\tau) = \int_{-\infty}^\infty\overline{f(t)}g(t+\tau) dt
$$
hence
$$
\text{corr}(f,g)(0) = \langle f ,g \rangle
$$
and uncorrelation means orthogonality.
The same applies for discrete time signals in $l_2$