How can I prevent Homebrew from installing a duplicate of software I installed through other means?
Take the path that you used for --prefix= during your ./configure and sym-link that into the Cellar/ directory that Homebrew is using. Homebrew will see these directories and see the dependencies as satisfied (unless you need to have them compiled with or without certain features to make Homebrew's formulas compile correctly). If your manually-compiled deps aren't compiled correctly, you'll probably get errors out of Homebrew.

ln -s /path/from/prefix/ /usr/local/Cellar/path/from/prefix

(Note the trailing slash on the first path, and lack of trailing slash on the second. OS X can be picky about these slashes when creating sym-links.)

This is the wrong way to do it, you should just be using --ignore-dependencies coq — that's what it's there for.