ICA - Statistical Independence & Eigenvalues of Covariance Matrix I am currently creating different signals using Matlab, mixing them by multiplying them by a mixing matrix A, and then trying to get back the original signals using FastICA.

So far, the recovered signals are really bad when compared to the original ones, which was not what I expected.

I'm trying to see whether I'm doing anything wrong. The signals I'm generating are the following:

s1 = (-x.^2 + 100*x + 500) / 3000; % quadratic
s2 = exp(-x / 10); % -ve exponential
s3 = (sin(x)+ 1) * 0.5; % sine
s4 = 0.5 + 0.1 * randn(size(x, 2), 1); % gaussian
s5 = (sawtooth(x, 0.75)+ 1) * 0.5; % sawtooth




One condition for ICA to be successful is that at most one signal is Gaussian, and I've observed this in my signal generation.

However, another condition is that all signals are statistically independent.

All I know is that this means that, given two signals A &amp; B, knowing one signal does not give any information with regards to the other, i.e.: P(A|B) = P(A) where P is the probability.

Now my question is this: Are my signals statistically independent? Is there any way I can determine this? Perhaps some property that must be observed?

Another thing I've noticed is that when I calculate the eigenvalues of the covariance matrix (calculated for the matrix containing the mixed signals), the eigenspectrum seems to show that there is only one (main) principal component. What does this really mean? Shouldn't there be 5, since I have 5 (supposedly) independent signals?

For example, when using the following mixing matrix:

A =

0.2000    0.4267    0.2133    0.1067    0.0533
0.2909    0.2000    0.2909    0.1455    0.0727
0.1333    0.2667    0.2000    0.2667    0.1333
0.0727    0.1455    0.2909    0.2000    0.2909
0.0533    0.1067    0.2133    0.4267    0.2000


The eigenvalues are: 0.0000 0.0005 0.0022 0.0042 0.0345 (only 4!)

When using the identity matrix as the mixing matrix (i.e. the mixed signals are the same as the original ones), the eigenspectrum is: 0.0103 0.0199 0.0330 0.0811 0.1762. There still is one value much larger than the rest..

Thank you for your help.

I apologise if the answers to my questions are painfully obvious, but I'm really new to statistics, ICA and Matlab. Thanks again.

EDIT

I have 500 samples of each signal, in the range [0.2, 100], in steps of 0.2, i.e. x = 0:0.1:100.

Also, given the ICA Model: X = As + n (I'm not adding any noise at the moment), I am referring to the eigenspectrum of the transpose of X, i.e. eig(cov(X')).

UPDATE

As suggested (refer to comments), I tried FastICA on only 2 signals. The results were quite good (see pic below). The mixing matrix used was A = [0.75 0.25; 0.25 0.75]. However, the eigenspectrum  0.1657 0.7732 still showed only one main principal component.

My question therefore boils down to the following: What function/equation/property can I use to check whether a number of signal vectors are statistically independent?
Signals 3 and 5 appear to be quite correlated -- they share their first harmonic. If I were given two mixtures of those, I wouldn't be able to separate those, I'd be tempted to put the common harmonic as one signal and the higher harmonics as a second signal. And I would be wrong! This might explain the missing eigenvalue.

Signals 1 and 2 do not look independent too.

A quick and dirty "sanity-check" for the independence of two series is to do a (x, y) plot of one signal against the other:

plot (sig3, sig5)


and then to do the same (x, y) plot with one signal shuffled:

indices = randperm(length(sig3))
plot(sig3(indices), sig5)


If the two plots have a dissimilar look, your signals are not independent. More generally, if the (x, y) plot of the data shows "features", disymmetries etc, it's a bad omen.

Proper tests for independence (and those are the objective functions used in the ICA optimization loop) include, for example, mutual information.

ICA is recovering the most independent signals a linear mixing of which yields your input data. It will work as a signal separation methods and recover the original signals only if those were maximally independent according to the optimization criterion used in your ICA implementation.