SQL Server Linked server query problem (group by)
The latest full client will work fine with 10g. 
I have experienced similar problems using anything other than the full client, even though according to Oracle the others should work. 
Just make sure that you completely uninstall all elements of the previous installs as having multiple clients/versions can create another set of problems.