C++ based FIR filter design using GNU Octave remez function I am designing a system where I want a FIR filter coefficient generator feature. 
I try the remez.cc source file provided in the Octave Signal package.

The source code for remez.cc is here. I have removed the octave interface part (line 757 - end) and the library header (line 34) and created a simple header with only remez function at line 592 then compiled the source code into a static library.

#pragma once
// remez.h
/********************
* remez
*=======
* Calculates the optimal (in the Chebyshev/minimax sense)
* FIR filter impulse response given a set of band edges,
* the desired response on those bands, and the weight given to
* the error in those bands.
*
* INPUT:
* ------
* int     numtaps     - Number of filter coefficients
* int     numband     - Number of bands in filter specification
* double  bands[]     - User-specified band edges [2 * numband]
* double  des[]       - User-specified band responses [numband]
* double  weight[]    - User-specified error weights [numband]
* int     type        - Type of filter
*
* OUTPUT:
* -------
* double h[]      - Impulse response of final filter [numtaps]
* returns         - true on success, false on failure to converge
********************/
int remez(double h[], int numtaps,
          int numband, const double bands[],
          const double des[], const double weight[],
          int type, int griddensity);


Then I followed an Octave example for filter design and implemented a C++ main function for experimenting how to call the remez function. Here is my main function.

#include &lt;remez.h&gt;
#include &lt;cmath&gt;
#include &lt;vector&gt;
#include &lt;iostream&gt;

using namespace std;

int main()
{
    int numtaps = 52;
    vector&lt;double&gt; h(numtaps);
    vector&lt;double&gt; bands = { 0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9 };
    int numbands = bands.size() / 2;
    vector&lt;double&gt; des = { 1, 1, 0, 0, 0.5, 0.5, 0, 0, 1, 1 };  // designed response for each band

    double ripple1 = 1 - pow(10, -0.3 / 20);
    double att2 = pow(10, -60 / 20);
    double ripple3 = (1 - pow(10, -0.2 / 20)) * 0.5;
    double att4 = pow(10, -70 / 20);
    double ripple5 = 1 - pow(10, -0.1 / 20);
    vector&lt;double&gt; weight = { 1 / ripple1, 1 / att2, 1 / ripple3, 1 / att4, 1 / ripple5 };

    int err = remez(h.data(), numtaps, numbands, bands.data(), des.data(), weight.data(), 1, 16);
    for (auto &amp;c : h)
        cout &lt;&lt; c &lt;&lt; ", ";
    cout &lt;&lt; endl;
    cout &lt;&lt; "error code: " &lt;&lt; err &lt;&lt; endl;
    return 0;
}


This function call produce some weird coefficients. All values inside h vector are 10e144 something. It is clearly an overflow so I debugged the remez.cc and traced what cause this.

The remez function documentation specified that des array should be of size numband. The des array in remez function is passed to CreateDenseGrid function on line 86 for grid generation. The documentation for CreateDenseGrid on line 73 indicate that its parameter des should be size of 2*numband. Clearly since my des array is only of size numband some out of bound memory access happened on line 110.

I changed the des vector to be {1, 1, 0, 0, 0.5, 0.5, 0, 0, 1, 1} and tries again. The coefficients looks nicer, but when I plot the frequency response in Octave it is not anything close to my intended filter design.

I get this weird frequency response:


Anyone could help me with this issue? Is anything wrong with my remez function call? Any suggestion is appreciated here. Thank you very much.

By the way. I am using vs2015, windows7 64-bits, Debug X86 build for compiler settings.
There are some known bugs (such as bug #38134 and bug #48078 on savannah) with the current implementation of remez found in Octave's signal package (see remez.cc on sourceforge), not to mention that the source to link to seems to lag a little from that latest version (you are thus lacking some critical fixes that have been included to address various stability and convergence issues).

Since you appear to be reasonably comfortable with modifying sources, you may want to try out the latest remez.cc source with the submitted patch attached to bug #38134, or alternatively the newly submitted firpm replacement to remez.