Do I need to remove all carpet adhesive residue before using a floor leveler?
Assuming this is a floating vinyl floor (as originally posted), you need to scrape away the high spots, lay down an underlayment that has vapour barrier with foam and optionally felt to provide a smoothing cushion.
For planks, there's no need to prime or scrape bare.
As for flatness and smoothness, the manufacturer will specify it, but usually anything less than 1/16in is not a problem, and some bowing and bending over several multiples of the plank length or its width is tolerated quite well.
If you are flattening with self-leveling cement, for whatever reason, you have to scrape the concrete bare, entirely.
As for a concrete or cement primer, read the label on the self-leveling cement: some don't need primer and they are more expensive. Others require primer which you can add-in or roll-on to your surface.
Which one you choose depends a bit on how thick the layer will be: anything over 1/4 (very roughly) and it's more economical to roll it on first, otherwise you're just mixing very expensive cement.