Carbon steerer with expander plug under the stem, should I be worried?
Should you be worried, I wouldn't be. The alloy expander will stop the carbon steerer from being crushed. Remember the max torque on the stem bolts will likely be less than 6nm. (Use appropriate retaining paste and don't over tighten it!) 

As you mentioned, a lot of people (me included) will recommend a 5mm spacer on top of the stem, with the steerer a bit longer, to be extra safe. 

But plenty of people refuse to have spacers above the stem for various reasons, mainly aesthetics, and they are fine, assuming the expander insider the steerer is of good quality.