Is there a difference between a chart and map?
Colloquially, I would say not really, by definition yes.
I think the simplest way to put is is:

All charts are maps but not all maps are charts.

Websters defines a chart as 

: map: such as
a : an outline map exhibiting something (such as
climatic or magnetic variations) in its geographical aspects
b : a map for the use of navigators

while they define a map as

a : a representation usually on a flat surface of the whole or a part
of an area
b : a representation of the celestial sphere or a part of it
c : a diagram or other visual representation that shows the
relative position of the parts of something

In other words, maps are outlines of part or all of something while charts contain potentially more information like that needed to navigate or identify airspace. Interestingly according to the definition, if used for navigation its a chart.