Import CSV with Single Quoted data
Part of the problem here is that there's no standard specification for CSV.

Added to that, Microsoft does not have much incentive to ensure interoperability between their version of CSV and Drupal's version, and so it is not surprising that they do not offer any (direct) way to encapsulate the contents of cells with single quotes within double quotes, so you've got to do something a bit more complicated if you want Excel to output Drupal-friendly CSV files.

Or, since Drupal is telling you where the problem is, you can do it manually.  If you only have a few rows with apostrophes, then that may not be a big deal.  If you have a lot of rows in your CSV with apostrophes or single quotes, then you've got a bit of pain.

Alternately, if you've got Python on your machine, or are willing to install it, this python script to add double-quotes to CSVs may work for you, and may be much easier than dealing with Excel macros.  It all depends on the languages and language environments you're comfortable with, however.

One might argue that Drupal (or more specifically the Feeds module) should support Microsoft's CSV, and thus handle apostrophes gracefully when not wrapped in double quotes.  If you are of this opinion, you may want to file a request with the Feeds project.  Since there is no (standard) CSV spec, this can't be considered a bug (AFAIK), but it does seem like a useful feature that they might want to add.