title: Filtering Invalid Byte Sequences for UTF8 Encoding in C# for a PostgreSQL Database
body: <p>I have been searching and attempting  a lot of different solutions to this problem, but have not found an answer yet. Basically, I am inserting data from an ODBC connection into a PostgreSql database using the COPY query, but the COPY query stops and returns this error...</p>
  
  <pre><code>ERROR:  invalid byte sequence for encoding "UTF8": 0x92
  CONTEXT:  COPY [TableName], line 1: "189572|1-00-1202|1-|00-|1202||AP||1...
  STATEMENT:  COPY [TableName] FROM STDIN (DELIMITER '|', NULL '')
  </code></pre>
  
  <p>The ODBC connection driver is set as ANSI and the Sybase/Advantage database it is pulling from is encoded as Unicode. To put this error message in perspective, out of 360,000 entries it only trips up on one of them. The issue is the COPY query stops and does not complete even if there is one error.</p>
  
  <p>Below is my C# code that is extracting the data...</p>
  
  <pre><code>OdbcDataReader reader = test.ExecuteReader();
  int rowCount = reader.FieldCount;
  
  while (reader.Read())
  {
  
      for (int i = 0; i &lt; rowCount; i++)
      {
          dataEntry = dataEntry + reader[i].ToString() + "|";
      }
  
      dataEntry = dataEntry.Trim().Substring(0, dataEntry.Length - 1).Replace("\r",string.Empty).Replace("\n", string.Empty);
  
      UTF8Encoding utf8 = new UTF8Encoding();
  
      var raw = utf8.GetBytes(string.Concat(dataEntry,"\n"));
  
      copy.CopyStream.Write(raw, 0, raw.Length);
      dataEntry = "";
  }
  </code></pre>
  
  <p>Basically, I am looking for a way to filter out invalid UTF8 byte sequences in C# and either remove them or replace them with a different character so the COPY query doesn't report an error. Any help is greatly appreciated. Thank you.</p>