How to Deny A Project?
I thought, sure, why not and had the disclaimer that if the project is viable I'm on. 


It sounds like there may be some methodological issues with his proposed work, but setting that aside, it is not incumbent on you to work on a project that is not of interest to you, a fortiori if you think that his work will be driven by biases relating to the topic.  From your description, it appears that you have already told him that your participation is contingent on this project being viable.  In view of that, I think you at least owe him an explanation if you pull out, and an apology if this comes from some reason other than the viability of the project.  I see two broad options for pulling out:


(1) Tell him that you think his project has some methodological problems (e.g., tenuousness of inferring race from name, etc.) and that you therefore don't think it is viable research.  In this case your choice is consistent with your previous representations to him, so you do not owe him any apology.
(2) Tell him that you have changed your mind, and you have decided that --regardless of its viability-- the project is not something of interest to you.  Be up-front about this and apologise for changing your mind - acknowledge that you previously said you'd participate, but let him know that you have changed your mind.


Either way, the best thing in these cases is to raise this as early as you can, be up-front in your position, and acknowledge any previous representations you made to him.  If you wish him luck with his research then this is likely to smooth over any discomfort.