Is this sampling frequency right? I designed a Butterworth second order bandstop filter on Matlab with the following parameters:


Notch frequency: 150 Hz (the frequency I want to remove)
Bandwidth = 8 Hz, that is 
Fc1=146 Hz
Fc2=154 Hz
Fs (sampling frequency) = 10,000 Hz


According to Nyquist Criterion, 0.5xFs > Fm 
where Fm is my signal’s highest frequency. Let's say that my signal frequency ranges from 0 to 3 kHz, so my sampling frequency should be 6 kHz or higher. As you saw, I chose a samplig frequency of 10 kHz. 

MATLAB gave me the filter coefficients for my difference equation, that is 

y[n]=b0x[n]+b1x[n−1]+b2x[n−2]−a1y[n−1]−a2y[n−2] (IIR Filter).

I implemented this equation in a STM32F767 Nucleo Board. The input signal goes to the ADC, the signal is processed and then go to the DAC for output. I used a timer interrupt every 100 us for this. I mean, the process takes place in the timer interrupt loop. 

**

The problem is this:
My filter works, but it's no filtering the frequency I chose to filter. I wanted to remove the 150Hz, but it's removing a frequency of 298~300Hz.

** 

What could be happening here? Does the sampling frequency have to do with this? Should I choose a lower sampling frequency? If it's needed I can attach my C code as well as the MATLAB design. But this is the basic idea:

struct difference_equation
{
  // coefficients
  double b0;
  double b1;
  double b2;
  double a1;
  double a2;
  // input history
  double x1;
  double x2;
  // output history
  double y1;
  double y2;

  double process(double x0)
  {
    double y0 = b0 * x0 + b1 * x1 + b2 * x2
      - a1 * y1 - a2 * y2;

    x2 = x1;
    x1 = x0;

    y2 = y1;
    y1 = y0;

    return y0;
  }
}
The STM32 is a 32bit micro-controller and all of your processing seems to be taking place over a double precision data type.

Doubles are 64bit "numbers" but not in the straightforward sense that 64bit integers are. Therefore, mathematical operations on doubles can take some additional steps (see page 47 onwards) on CPUs that are not equipped with special hardware to handle floating point operations. If the CPU does not have an FPU then mathematical operations are handled in software which means that they take more CPU cycles to complete (For example, you can try double precision multiplications on an Arduino (like Uno), it's going to be fun).

The STM32F767 does have (see page 13) a Floating Point Unit, which means that it can potentially perform double precision operations faster than when these operations are implemented in software and take up CPU cycles. Notice that the FPU is better than not having it but it still has its own limits, for indicative timings see page 16 on this document

BUT, this has to be enabled (see page 20).

Therefore, my guess is that your interrupt routine is striving to fire at the intended 10kHz but it doesn't complete before $\frac{1}{10000}$ seconds. So, your effective sampling frequency is not 10kHz and the whole system is transposed to a different frequency range.

You might want to check if the FPU is enabled at double precision or move to fixed point numbers which every CPU will happily crunch. With fixed point numbers, you have another problem which is that your filter might go out of spec because of the rounding that is inevitable due to the reduction in precision. But these are aspects that you have to manage in your solution.

Hope this helps.