Concept of Orthogonality and Orthonormality
Orthogonal means that the inner product is zero. For example, in the case of using dot product as your inner product, two perpendicular vectors are orthogonal. Orthonormal means these vectors have been normalized so that their length is 1. 

Orthogonal vectors are useful for creating a basis for a space. This is because every point in the space can be represented as a (linear) combination of the vectors. So for example in 3D space, x=[0,0,1]  y=[0,1,0] and z=[1,0,0] form an orthornormal basis. No component of x can be represented with a component of the other vectors. This is because they are linearly independent

This is a different type of independence to statistical dependence however.

Correlation is a different notion. There are different ways of representing the correlation of two vectors (random variables X and Y).

$corr(X,Y) = \frac{cov(X,Y)}{\sigma_x \sigma_y} = \frac{E[X - \mu_x]E[Y - \mu_y]}{\sigma_x \sigma_y}$

Covariance is a measure of how two variables change together.

Another measure of correlation is cross-correlation. This is measure of the similarity of two functions (or vectors). 

These concepts ares used in many different ways in image processing.

For example cross-correlation is used in template matching. When looking for a small image inside a much larger image, the small template image is 'slided' over the large image and the cross-correlation is computed for each position. Locations with a high cross-correlation are likely to contain the image we are looking for.

The concept of Linearly independent component vectors is used in principal component analysis (PCA). PCA takes a cloud of points in space and calculates a set of orthogonal basis vectors to represent them.  

Independent component analysis (ICA) is used for separating mixed signals. ie: separating a speaker at a noisy cocktail party. ICA uses properties of the signal correlation to separate the signals.