ESD - Is it safe to connect yourself to an electrical outlet?
If your plug has a correctly wired earth pin, then you can use that to ground yourself.

However before you do, it is worth noting that all reputable ESD wrist bands contain an embedded 1MOhm resistor. This is done primarily so that if you accidentally connect yourself to mains voltage (either through what you are handling, or through a wiring fault in the plug) the resistor limits the current that can pass through you, and so protects you from certain death.

I would suggest buying an ESD wrist strap. If you make your own, be sure to include a 1M resistor. 

To be fair though even if you don't include the resistor, as long as you are absolutely sure you have connected yourself to the earth pin and nothing else, it will not do anything worse than touching the back panel on your PC (or the case of the PSU) because those are generally tied directly to earth on the outlet anyway. If touching your PC gives you an electric shock, consult an electrician.

As an alternative to the earth pin on an electrical outlet, you can also use things like metal plumbing pipes as these are typically earth bonded to the electrical system earth in the house, or go down physically into the ground.