How to provide the ability to upload media to the server? I want to provide the ability to upload PDFs/mp3s etc. to the server.

What is the best way of achieving this?

I've seen the media plugin but it seems that replaces the upload field rather creating a stand-alone media area? Or maybe it does?
I think Media is a great module unless if your client will upload and use the same PDFs/MP3 once a time.

But if your client plan to use at severals places the same Media, so Media module is for you.

Also, give a look at Scald, seems to be a promise module.