Deck the halls with bubble wrap?
I would select non-flammable insulating materials such as rock-wool, or fiberglass (fully puffed up; mashing it badly hurts its performance).  To use these, you will need some sort of framing to hold it up.  It can't just be taped to a wall.
It's not out of the question to frame a room inside a rental house; I've seen it done. However this is the wrong year to be buying lumber and building supplies: COVID shut down the logging and milling earlier this year, meanwhile everyone stuck at home has gone on a remodeling binge, meanwhile new-home construction remains a very strong market.
However, you really need to work the normal insulation priority list: sealing of doors and windows first, window treatments second, then walls pretty much last.