How can we get CA's public key?
You can search for root certificates of a given CA.

E.g.:


http://www.symantec.com/page.jsp?id=roots, but this page is served over plain HTTP so maybe you shouldn't trust it!
or https://www.mozilla.org/projects/security/certs/included/.


From there when you want to check a certificate you can check whether it belongs to/was signed by a root CA you trust. If you trust it, then you can get its public key.

More commonly as CodesInChaos said, trusted certificates are shipped with your OS/browser.