mysql works, mysqldump doesn't
When reading the configuration file, the mysql command line client only reads configuration directives found in the [client] and [mysql] sections, while mysqldump only uses configuration directives found in the [client] and [mysqldump] sections.

If your socket directive in the config file is in the [mysql] section but not the [mysqldump] section then you should move it to the [client] section, and this should solve your problem.

So where does /var/run/mysqld/mysqld.sock come from?  I remember very distinctly, once upon a time, beating my head against the wall repeatedly over that one.

It turns out, Ubuntu compiles their MySQL components from source, and when they do that, they compile them to use /var/run/mysqld/mysqld.sock unless the configuration file specifies otherwise or the socket is overridden on the command line... so if your socket directive isn't in [client] that would be one explanation of this behavior.