Using BCP to import data to SQL Server while preserving accents, Asian characters, etc
Question: I'm trying to import a PostgreSQL dump of data into SQL Server using bcp. I've
written a Python script to switches delimiters into '^' and eliminate other
bad formatting, but I cannot find the correct switches to preserve unicode
formatting for the strings when importing into SQL Server.

In Python, if I print out the lines that are causing me trouble, the row looks
like this with the csv module:

    
    
    ['12', '\xe4\xb8\x89\xe5\x8e\x9f \xe3\x81\x95\xe3\x81\xa8\xe5\xbf\x97']
    

The database table only has 2 columns: one `integer`, one `varchar`.

My statement (simplified) for creating the table is only:

    
    
    CREATE TABLE [dbo].[example](
        [ID] [int] NOT NULL,
        [Comment] [nvarchar](max)
        )
    

And to run bcp, I'm using this line:

    
    
    c:\>bcp dbo.example in fileinput -S servername -T -t^^ -c
    

It successfully imports about a million rows, but all of my accented
characters are broken.

For example, "Böhm, Rüdiger" is turned into "B+¦hm, R++diger". Does anyone
have experience with how to properly set switches or other hints with bcp?

**Edit** : `varchar` switched to `nvarchar`, but this does not fix the issue.
This output in Python (reading with CSV module):

    
    
    ['62', 'B\xc3\xb6hm, R\xc3\xbcdiger']
    

is displayed as this in SSMS from the destination DB (delimiters matched for
consistency):

    
    
    select * from dbo.example where id = 62
    62;"B├╢hm, R├╝diger"
    

where in pgAdmin, using the original DB, I have this:

    
    
    62;"Böhm, Rüdiger"
    


Answer: You may need to modify your BCP command to support wide character sets (note
the use of -w instead of -c switch)

    
    
    bcp dbo.example in fileinput -S servername -T -t^^ -w
    

[BCP documentation reference](http://msdn.microsoft.com/en-
us/library/ms162802.aspx)

See also <http://msdn.microsoft.com/en-us/library/ms188289.aspx>

