How to setup a ssh tunnel that works with P2P and filtered http from OS X to a linux machine over SSH?
Transmission will likely try to connect to the tracker using UDP which cannot be transmitted through the SSH proxy. You may or may not be able to force uTorrent to only use TCP. Even then it's unlikely that this will work.
Also this setup doesn't really make a lot of sense. You'd be much better off by downloading/seeding your torrents directly on that Linux box. rtorrent or transmission-daemon comes to mind as a viable solution.
You do have a lot of redundant options in your ssh command. You can strip it down to ssh -NC user@server.tld -D 1080 though I'd suggest you put all the config info into ~/.ssh/config anyway. You may increase the security of your connection by only binding your SOCKS proxy to the loopback by using -D localhost:1080. You do not need the reverse tunnel for your question and may even be substantially violating a company policy with that.