Determining voltage of fuse
I have a 20A fuse lying around and a 7.5A fuse that I took out the car. 


It probably is an ATO type fuse similar to this?




  The car battery charger states that it needs a 5A 32V fuse, 


In general a replacement fuse should at least


have a voltage rating equal or higher to the specified voltage
have a current rating equal to or slightly smaller than the specified current



  so I'm guessing the 7.5A will be fine as a once off? 


7.5A may not properly protect equipment from damage when 5A is specified, but it will probably OK for the charger. 

The common ATO Automotive Blade Fuses are normally rated for 32V. The voltage rating matters with respect to arcing.