Removing old baseboard heaters and thermostat
I will assume you mean the thermostats that are on the wall separate from the baseboard heaters, not the ones built into the heaters. These should be installed on top of an electrical box.  You can remove the wires from the thermostat, cap them with wire nuts, tuck them into the box, and install a blank cover plate over the box.  When selecting wire nuts, read the instructions and find wire nuts that are rated for a single wire of the size you're capping.  Put a wrap of tape on there for good measure.  

Hopefully the baseboard units were installed on top of boxes - sometimes they are installed with the wire coming through a hole in the drywall secured in the heater.  You shouldn't just cap these wires and tuck them in the wall, if there's no box I'd install a box just to hold the decommissioned wires.  

You can also disconnect the wires from the breakers in the panel if there's nothing else left on those circuits.  You can cap the wires the same way in the electrical panel.  

Some would say that if the wires are disconnected from the breaker, you don't have to worry about capping the wires in boxes, but in my opinion it's better to cap them in a box and maybe use them in the future, and if someone does connect them to power in the future, you don't create any hazard.