Intel Mac mini - maximum RAM Upgrade
For starters, OS X doesn't have the silly 64/32-bit dichotomy that Windows and Linux do. For most purposes, you can consider recent versions of OS X to be both 32- and 64-bit at the same time. The limiting factor in almost all cases of upgrading Mac RAM is the hardware chipset. 

For your Mac mini 4,1: my copy of Mactracker says that the maximum RAM is 8GB. Your five-gig setup should work fine.