How can I report security vulnerabilities for open source OSX applications?
You can contact Apple about this at product-security@apple.com (or you can open a Radar report if you're a developer), or you contact the maintainer of the package.
Often the mail addresses can be found in the README (or AUTHORS) of the source code, or on the project's website.

Yes - both Apple (specifically) and the open source developers (in general) do reference CVE in patch and security emails and participate using that mechanism for tracking reported vulnerabilities.