How to remove symbol from file I am using ssis to generate some .dat files which are then sent to linux server for automated load. It looks as windows add some symbol character in file which is not easily identifiable first but if i open file on linux server i can see that symbol.

For eg. on our side file looks fine but on linux side when opened I see ÿþ is added as first charater in file.  This is UTF-16 (LE) little-endian character represented as Char(255) and Char(254) in decimal. I tried to use replace to remove this symbol but didn't work
For eg. Select replace(replace('ÿþ01CAP   00820130208',char(255),''),char(254),'') 

This works fine for string but there is nothing to replace in the file generated on windows side. Once it is uploaded to linux that symbol is sticked into it..
Any ideas ? What could be wrong here ?
I tried to use diff collation in query that generates this file but no luck

Thanks,
NJ
Unicode files should have BOM, is the recommended and accepted way, specially for LE:


  In UTF-16, a BOM (U+FEFF) may be placed as the first character of a
  file or character stream to indicate the endianness (byte order) of
  all the 16-bit code units of the file or stream. If the 16-bit units
  are represented in big-endian byte order, this BOM character will
  appear in the sequence of bytes as 0xFE followed by 0xFF. This
  sequence appears as the ISO-8859-1 characters þÿ in a text display
  that expects the text to be ISO-8859-1.
  ...
  "The UTF-16 encoding scheme may or may not begin with a BOM. However, when there is no BOM, and in the absence of a higher-level protocol, the byte order of the UTF-16 encoding scheme is big-endian."


'linux' is a very generic term. You must be using some application for processing, and if the application is not recognizing the BOM it means is a bad application. Ditch it for something better.