How do I check my Mac's RAM?
I've had bad RAM that still allowed the machine to boot - a stick of memory in my Mac Pro apparently failed gracefully overnight. 

The first step would be to check system profiler to see if the memory is being recognized. If indeed it is, there's a version of MemTest for OS X: http://osxdaily.com/2011/05/03/memtest-mac-ram-test/