Uncovering data from noise I've recently started experimenting with DSP, specifically, understanding digital signals that are generated by some small applications I am creating. For this quest, I've decided to use The Scientist and Engineer's Guide to Digital Signal Processing. While the provided material is beyond exceptional, the more I read the more confused I become as to how I could apply different forms of DSP in my "computer based" experiments.

Here is what I am currently working on:

I have a capacitative sensor. When the sensor is stretched, the capacitance increases. Contrary is true when it contracts. Image below displays this characteristic:



An interesting observation I noticed is that when I stretch the sensor, while bending it, my signal is buried in noise. This is evident between 800 and 1000 samples in the above photo. Because I am experiencing this issue, I want to uncover the stretch/contraction of the sensor from this noise; hence my goal of studying DSP.

An interesting comment made by the author, is that when ever we deal with Digital Data we will always want to analyze our Time Domains using DFT. The most efficient algorithm to do this today is FFT. I've also concluded ( high level ) that using the mentioned algorithm is ideal as it will give me a sense of what "sin waves" generate the "digital wave" I am presented with. Hence, I should be able to parse out "sin waves" of interest to re-construct missing points of interest hidden in noise.

Now, while I understand that the change to the frequency domain can tell me that "a spike has occurred", what I am really interested in is re-constructing my wave to be less noisy. Hence, I'd like to derive out a consistent wave form between 800 - 1000, alike I have between 400 - 600. 

Questions


Since my understanding is still fairly low, is DFT and FFT really the solution to my problem? 
Would it be logical to take the FFT , filter out outliers that are generated and cause noise, and re-construct the time domain signal back ( with the results that were acceptable ) using iFFT? That should, in theory, give me back a time domain with less noise, no?
Is there a better section of DSP that I should read that can help me analyze the decouple noise from interesting data in a time domain?
Since my understanding is still fairly low, is DFT and FFT really the solution to my problem?


All the FFT (or DFT) really gives you is a way of visualizing the data in the frequency domain.

Your data may be good for this, except that there will be a large spike at "DC" (the zero frequency bin) because your data has a non-zero mean.


  Would it be logical to take the FFT, filter out outliers that are generated and cause noise, and re-construct the time domain signal back ( with the results that were acceptable ) using iFFT? That should, in theory, give me back a time domain with less noise, no?


If you know the approximate frequency of the variation you're after, yes, you can use the FFT (and IFFT) to filter out frequencies different from the band where the frequency will be.

One simple thing you could try without using an FFT is to use a DC blocker.  That will make your signal approximately zero mean, and will also smooth out some of the roughness you see. 


  Is there a better section of DSP that I should read that can help me analyze the decouple noise from interesting data in a time domain?


One place to start might be chapter 15 of that book.

I suspect you'll want to design a bandpass filter around the frequencies of interest and filter out everything except this band. You will still see some effect of the noise, but it should be greatly reduced.

If that isn't good enough, then you may need to try more advanced techniques which don't appear to be covered by the book.