Does formal charge affect bond polarity?
When you compute formal charges you split bonding electrons evenly between bonded atoms. This does not account for differences in electronegativity. It is similar to the difference between formal charge and oxidation state, as well explained in the Wikipedia.
A better picture (closer to the real electronic distribution) is, as Pauling would probably suggest, intermediate to the two, assuming a hybrid between possible Lewis structures (covalent and ionic).
In any case, when considering bond polarity, electronegativity overrules formal charge.    On the other hand inductive effects should not be dismissed.
The OP presented an excellent counterexample: CO. Here the carbon has a very slight negative charge relative to oxygen, as predicted based on the computation of formal charges.
I am not an expert on the development and application of electronegativity scales, but suffice it to say that these scales are (1) empirical and (2) provide useful guidelines. The original Pauling formulation of electronegativity cannot account for all possible bonding schemes. Case in point is the modification of the electronegativity scale to account for orbital hybridization, the following data via the Wikipedia:




Hybridization
χ (Pauling)1




C(sp3)
2.3


C(sp2)
2.6


C(sp)
3.1


'generic' C
2.5




Based on this modified scale the electronegativity of oxygen, 3.44, is no longer much larger than that of the carbon atom in carbon monoxide.
References

Data found in Wikipedia, original reference: Fleming, Ian (2009). Molecular orbitals and organic chemical reactions (Student ed.). Chichester, West Sussex, U.K.: Wiley.