title: Replace Unicode character "ï¿½" with a space
body: <p>I'm a doing an massive uploading of information from a .csv file and I need replace this character non ASCII &quot;ï¿½&quot; for a normal space, &quot; &quot;.</p>
  <p>The character &quot;ï¿½&quot; corresponds to &quot;\uFFFD&quot; for C, C++, and Java, which it seems that it is called <a href="https://www.utf8-chartable.de/unicode-utf8-table.pl?start=65280" rel="nofollow noreferrer">REPLACEMENT CHARACTER</a>. There are others, such as spaces type like <a href="https://www.utf8-chartable.de/unicode-utf8-table.pl?start=65279&amp;number=128" rel="nofollow noreferrer">U+FEFF</a>, <a href="https://www.utf8-chartable.de/unicode-utf8-table.pl?start=8287&amp;number=128" rel="nofollow noreferrer">U+205F</a>, <a href="https://www.utf8-chartable.de/unicode-utf8-table.pl?start=8192&amp;number=128" rel="nofollow noreferrer">U+200B</a>, <a href="https://www.utf8-chartable.de/unicode-utf8-table.pl?start=6158&amp;number=128" rel="nofollow noreferrer">U+180E</a>, and <a href="https://www.utf8-chartable.de/unicode-utf8-table.pl?start=8239&amp;number=128" rel="nofollow noreferrer">U+202F</a> in the C# official documentation.</p>
  <p>I'm trying do the replace this way:</p>
  <pre><code>public string Errors = &quot;&quot;;
  
  public void test(){
  
      string textFromCsvCell = &quot;&quot;;
      string validCharacters = &quot;^[0-9A-Za-z().:%-/ ]+$&quot;;
      textFromCsvCell = &quot;This is my text from csv file&quot;; //All spaces aren't normal space &quot; &quot;
      string cleaned = textFromCsvCell.Replace(&quot;\uFFFD&quot;, &quot;\&quot;&quot;)
        if (Regex.IsMatch(cleaned, validCharacters ))
          //All code for insert
        else
           Errors=cleaned;
           //print Errors
  }
  </code></pre>
  <p>The test method shows me this text:</p>
  <p><strong>&quot;This is myï¿½texto from csv file&quot;</strong></p>
  <p>I try some solutions too:</p>
  <p><strong>Trying solution 1: Using Trim</strong></p>
  <pre><code> Regex.Replace(value.Trim(), @&quot;[^\S\r\n]+&quot;, &quot; &quot;);
  </code></pre>
  <p><strong>Try solution 2: Using Replace</strong></p>
  <pre><code>  System.Text.RegularExpressions.Regex.Replace(str, @&quot;\s+&quot;, &quot; &quot;);
  </code></pre>
  <p><strong>Try solution 3: Using Trim</strong></p>
  <pre><code>  String.Trim(new char[]{'\uFEFF', '\u200B'});
  </code></pre>
  <p><strong>Try solution 4: Add [\S\r\n] to validCharacters</strong></p>
  <pre><code>  string validCharacters = &quot;^[\S\r\n0-9A-Za-z().:%-/ ]+$&quot;;
  </code></pre>
  <p>Nothing works.</p>
  <p>How can I replace it?</p>
  <p>Sources:</p>
  <ul>
  <li><p><em><a href="http://www.fileformat.info/info/unicode/char/0fffd/index.htm" rel="nofollow noreferrer">Unicode Character 'REPLACEMENT CHARACTER' (U+FFFD)</a></em></p>
  </li>
  <li><p><em><a href="https://stackoverflow.com/questions/3725444/">Trying to replace all white space with a single space</a></em></p>
  </li>
  <li><p><em><a href="https://stackoverflow.com/questions/1317700/">Strip the byte order mark from string in C#</a></em></p>
  </li>
  <li><p><em><a href="https://stackoverflow.com/questions/25954446/">Remove extra whitespaces, but keep new lines using a regular expression in C#</a></em></p>
  </li>
  </ul>
  <h1>EDITED</h1>
  <p><strong>This is the original string:</strong></p>
  <p>&quot;SYSTEM OF MONITORING CONTINUES OF GLUCOSE&quot;</p>
  <p><strong>in 0x... notation</strong></p>
  <p>SYSTEM OF0xA0MONITORING CONTINUES OF GLUCOSE</p>
  <h2>Solution</h2>
  <p>Go to the <em><a href="https://r12a.github.io/app-conversion/" rel="nofollow noreferrer">Unicode code converter</a></em>. Look at the conversions and do the <em>replace</em>.</p>
  <p>In my case, I do a simple replace:</p>
  <pre><code> string value = &quot;SYSTEM OF MONITORING CONTINUES OF GLUCOSE&quot;;
   //value contains non-breaking whitespace
   //value is &quot;SYSTEM OFï¿½MONITORING CONTINUES OF GLUCOSE&quot;
   string cleaned = &quot;&quot;;
   string pattern = @&quot;[^\u0000-\u007F]+&quot;;
   string replacement = &quot; &quot;;
  
   Regex rgx = new Regex(pattern);
   cleaned = rgx.Replace(value, replacement);
  
   if (Regex.IsMatch(cleaned,&quot;^[0-9A-Za-z().:&lt;&gt;%-/ ]+$&quot;){
      //all code for insert
   else
      //Error messages
  </code></pre>
  <p>This expression represents all possible spaces: space, tab, page break, line break and carriage return</p>
  <pre><code>[ \f\n\r\t\v​\u00a0\u1680​\u180e\u2000​\u2001\u2002​\u2003\u2004​\u2005\u2006​\u2007\u2008​\u2009\u200a​\u2028\u2029​​\u202f\u205f​\u3000]
  </code></pre>
  <p>References</p>
  <ul>
  <li><em><a href="https://developer.mozilla.org/en/docs/Web/JavaScript/Guide/Regular_Expressions" rel="nofollow noreferrer">Regular expressions</a></em> (<a href="https://en.wikipedia.org/wiki/MDN_Web_Docs" rel="nofollow noreferrer">MDN</a>)</li>
  </ul>