Can ICA be applied, when the number of mixture signal is less than number of source signal?
You are making a wrong assumption on the process. In ICA, the number of mixtures must be at least as many as the number of components. The paper you cite does in fact, acknowledge this:


  These observed signals from the red, green and blue color sensors are denoted by $x_1(t)$, $x_2(t)$ and $x_3(t)$ respectively, which are amplitudes of the recorded signals (averages of all pixels in the facial region) at time point $t$. In conventional ICA the number of recoverable sources cannot exceed the number of observations, thus we assumed three underlying source signals, represented by $s_1(t)$, $s_2(t)$ and $s_3(t)$.


The conversion $x_i^'=(x_i-\mu_i)/\sigma_i$ is merely centering and sphering of the data, which I explain in another answer on this site.



The cases considered in the paper are the noiseless ICA model and noisy ICA. In other words, the heart rate measurements considered at rest (not a pulseless model as you suggested) is the ICA model:

$$\mathbf{x}(t)=\mathbf{As}(t)$$

where $\mathbf{x}$ is the observed vector, $\mathbf{s}$ is the underlying component vector and $\mathbf{A}$ is the mixing matrix. 

On the other hand, heart rate measurements when in motion can be considered as

$$\mathbf{x}(t)=\mathbf{As}(t)+\mathbf{n}(t)$$

where $\mathbf{n}(t)$ is a noise vector (in this case the motions).