Powering BLDC motors using a Li-Po battery
The controllers can handle a voltage between 8 and 24 VDC. The motors are 12 V so I am guessing the controller contains some buck-boost converter. The datasheet does not say anything about this.

Very likely the controller does not have a buck-boost in it, at least to power the motor.  Far more likely is that there's a regulator down to 12V or 5V to power its electronics, but it would be just a buck, or a linear regulator.

Would this require any other components like converters between the battery and the controllers? Or will it be sufficient to just wire the battery directly to the controllers with maybe a fuse?

Just a fuse should be sufficient.  Depending on how the controllers are designed, you may want to keep the battery leads short (if you got RC batteries, and if you keep the plug that came on the battery, just make up a pigtail for the controller that's about that long).

I selected a 3s1p Li-Po battery with 11.1 V. would it be better to get more cells for a higher voltage since the motors are 12 V?

It should be fine.  You could probably go to 4s if you're careful about not driving too much current to the motors.  With 4s they could run a bit fast if you drove them at full speed.  If they had the opportunity, this would make them suffer from shortened bearing life -- but you're not going to be running this day in and day out (I assume).