Can I cover receptacles with solid metal cover without removing receptacle?
Locking Receptacle Cover
Use something like this locking cover:

It has a hole for a padlock. Lock it when you're away. Unlock when you need it.
This particular one (and many others) specifically weatherproof for outdoor use since that is a common application - prevent your neighbor from using your electricity - but it will work fine indoors too.
This is also an in-use cover. So you could have a device plugged in even while this is locked. For example, you could have an appliance that should stay plugged in while keeping the other receptacle of the duplex from being used.