title: Comparison with NaN using AVX
body: <p>I am trying to create a fast decoder for <code>BPSK</code> using the <code>AVX</code> intrinsics of Intel. I have a set of complex numbers that are represented as interleaved floats, but due to the <code>BPSK</code> modulation only the real part (or the even indexed floats) are needed. Every float <code>x</code> is mapped to <code>0</code>, when <code>x &lt; 0</code> and to <code>1</code> if <code>x &gt;= 0</code>. This is accomplished using the following routine:</p>
  
  <pre><code>static inline void
  normalize_bpsk_constellation_points(int32_t *out, const complex_t *in, size_t num)
  {
      static const __m256             _min_mask = _mm256_set1_ps(-1.0);
      static const __m256             _max_mask = _mm256_set1_ps(1.0);
      static const __m256             _mul_mask = _mm256_set1_ps(0.5);
  
      __m256                          res;
      __m256i                         int_res;
  
      size_t i;
      gr_complex                      temp;
      float                           real;
  
      for(i = 0; i &lt; num; i += COMPLEX_PER_AVX_REG){
              res = _mm256_load_ps((float *)&amp;in[i]);
  
              /* clamp them to avoid segmentation faults due to indexing */
              res = _mm256_max_ps(_min_mask, _mm256_min_ps(_max_mask, res));
  
              /* Scale accordingly for proper indexing -1-&gt;0, 1-&gt;1 */
              res = _mm256_add_ps(res, _max_mask);
              res = _mm256_mul_ps(res, _mul_mask);
  
              /* And then round to the nearest integer */
              res = _mm256_round_ps(res, _MM_FROUND_TO_NEAREST_INT |_MM_FROUND_NO_EXC);
  
              int_res = _mm256_cvtps_epi32(res);
  
              _mm256_store_si256((__m256i *) &amp;out[2*i], int_res);
      }
  }
  </code></pre>
  
  <p>Firstly, I clamp all the received floats in the range <code>[-1, 1]</code>. Then after some proper scaling, the result is rounded to the nearest integer. That will map all floats above <code>0.5</code> to <code>1</code> and all floats below <code>0.5</code> to <code>0</code>. </p>
  
  <p>The procedure works fine if the input floats are normal numbers. However, due to some situations at previous stages, there is a possibility that some input floats are <code>NaN</code> or <code>-NaN</code>. At this case, 'NaN' numbers are propagated through the <code>_mm256_max_ps()</code>, <code>_mm256_min_ps()</code> and all other <code>AVX</code> functions resulting to an integer mapping of <code>-2147483648</code> which of course causes my program to crash due to invalid indexing.</p>
  
  <p>Is there any workaround to avoid this problem, or at least set the <code>NaN</code> to <code>0</code> using <code>AVX</code>?</p>