How can I seal air leaks and otherwise winterize, as a renter?
For outlets, there are pre-cut foam gaskets that fit under the faceplates. You could leave those or remove when you go, as you prefer.

For windows that you will leave closed all winter, shrink film window sealing kits cover the whole window (best to put them all the way to the outside edge of the frame) and are fairly clear and non-obtrusive once shrunk, plus they seal the whole window (frame and all) if done that way. Peel them off in the spring.

I have used removable caulk on one building. Horribly stinky stuff, but it did peel off with no damage in the spring.