Uninstall PHP on Mac Mountain Lion
Don't uninstall the old php as it is /usr/bin and so supplied by Apple and so might affect the OS and Apple might reinstall it as part od an OS upgrade.

The binary install of php puts php in /usr/local/bin/php

To use it either use the full path or alter your PATH environment variable to have /usr/local/bin before /usr/bin