Why is Potassium in French and English not called Kalium?
(with thanks to @NilayGhosh, who provided the key source for much of this answer)

Short answer: Potassium was called Potassium (and Sodium Sodium) by Humphry Davy, who first isolated both; it was then renamed Kalium in Germanic countries because of a previous naming proposal by Klaproth, who was the first to show that potassium carbonate and sodium carbonate were different.

Until at least the Middle Ages no distinction was drawn between potash (potassium carbonate) and soda (sodium carbonate). Martin Heinrich Klaproth first distinguished them in 1797, and suggested the name 'kali' for the first and 'natron' for the second.

Metallic potassium and sodium were then isolated for the first time in 1807 by Sir Humphry Davy, by electrolysis of potassium hydroxide (KOH) and sodium hydroxide (NaOH) respectively. He named his discoveries Potassium and Sodium in the Bakerian lecture at the Royal Society on the 19th of November the same year:


  Potasium and Sodium are the names by which I have ventured to call the
  two new substances


(source)

(During the lecture he also apparently produced a nugget of metallic potassium and tossed it into a flask of water, "where the lump skittered around the surface of the water before exploding in lavender flames".)

This paper was translated into German by Ludwig Wilhelm Gilbert (supposedly in Annalen der Physik, although I can't find the original paper in any of the volumes from  1809). According to the article on vanderkrogt.net Gilbert replaced Potassium with Kalium and Sodium with Natronium "in agreement with German nomenclature" (i.e. following Klaproth's suggested names). From this point on, generally speaking, English- and French-speaking countries followed Davy's naming system, while the Germanic countries followed that of Klaproth/Gilbert, as can be seen from the sidebar lists here (potassium) and here (sodium).

Regarding the atomic symbols mismatch: the international system of atomic symbols was devised by the Swedish chemist Jacob Berzelius, a rival of Davy's, who followed the naming convention used by Germanic countries. Although actually, when first published (1813, in Thomas Thomson's Annals of Philosophy), Berzelius did follow Davy's names and used the symbols Po and So. Within a year he had switched to K/Na. So since the first publication of Berzelius' system actually followed Davy and used Po/So, the real surprise is that we ended up using K/Na.