What is the main contributor of the color of honey?
I found that the color of honey depends on the presence of plant
pigments including carotene, xanthophylls, anthocyanins, flavonoids,
polyphenols, as well as amino acids and mineral salts

Yes, you are right about that. The color differences is due to the difference in levels of minerals, anti-oxidants, and other trace elements that are present. In particular, levels of polyphenols, flavonoids, and carotenoid have been observed to have high correlation with color. Polyphenols are found in high concentration in darker honeys, whereas flavonoids are found in higher concentrations in lighter honeys.
References

Antioxidant and antimicrobial capacity of several monofloral Cuban honeys and their correlation with color, polyphenol content and other chemical compounds by Jose M.Alvarez-Suarez Sara Tulipani, Daimy Díaz, Yadiley Estevez, Stefania Romandini, Francesca Giampieri, Elisabetta Damiani, Paola Astolfi, Stefano Bompadre, Maurizio Battino, Volume 48, Issues 8–9, August–September 2010, Pages 2490-2499, DOI: https://doi.org/10.1016/j.fct.2010.06.021
Total Phenol, Antioxidant Activity, Flavonoids, Anthocyanins and Color of Honey as Affected by Floral Origin Found in the Arid and Semiarid Mediterranean Areas by
Taha M. Rababah  Mohamad Al‐Omoush  Susan Brewer  Mohammad Alhamad  Wade Yang  Mohammad Alrababah  Abd Al‐Majeed Al‐Ghzawi  Muhammad Al‐ u'datt  Khalil Ereifej  Fawzi Alsheyab  Ranya Esoh  Ali Almajwal, Volume38, Issue3, June 2014, Pages 1119-1128, DOI: 10.1111/jfpp.12071