Calculating an incoherence property from sub-optimal sampling patterns
$\Phi$ is the matrix that represents the way you sample your signal $x$. Actually,  $\Phi$ can be an identity matrix with some rows eliminated, which means you are picking a subset of $x$.

$\Psi$ represents the basis that you choose to expand $x$: 
$$x = \Psi c$$
A very simple example is that $x$ is a cosine function (thus containing only one frequency component), then you can take $\Psi$ to be the Discrete Fourier (or discrete cosine) matrix and $c$ is just the coefficient of the specific frequency, and it is pretty sparse (all the coefficients on other frequencies will be zero). 

Then the compressed signal $b$ can be represented as:
$$b = \Phi \Psi c$$

Actually you can also view $\Phi \Psi$ as selection of the random rows of $\Psi$, then
one possible option of $\Phi$ and $\Psi$ is (in Matlab):

Psi = fft(eye(n)); 
p = randperm(n);
Phsi = Psi(p(1:m),:); % choose the first m rows from the permutation 


Psi is $\Psi$, and Phsi is $\Phi \Psi$.



For your case, yes you can use singular value decomposition on the image space. But I would prefer to 

[u,s,v]=svd(M'*M); % spectrogram
Psi = s(1:m,1:m) * v(1:m,:);
U = u(1:m,:);


which indicates that you select the m most magnitude on spectrum components. Columns in v' are the orthogonal vectors that map the frequency component with the corresponding magnitude to spectrum space. Please note that you don't need u as the basis, since it is always equivalent between u * s * v' * x = b, and s * v' * x = u' * b. As a result, the compressed form is converted to:
$$U'b = \Phi \Psi c$$

Hope it helps. Thanks