Is there a way to stop someone from giving me bitcoins to an address that I own I know I should be thankful for free money. But sometimes it might come from an undesirable source. Is there a way of preventing unwanted credits?

I suppose I could look at the block chain and pay it straight back...
No, there is no way to block people sending you bitcoins.  You could, however, send the bitcoins back to the sender.  In fact, you can redeem the transaction output created by the sender in a new transaction, created by you, back to the sender.  This would show that precisely these funds (not just funds to the same amount) were transferred back to the sender.