What does CA certs contains? public key of site or CA? I believe it contains public key but whose? the site like a bank or the certificate authority who has signed?
As per definition, digital certificate is a security document that binds public key to some entity. That is, any X.509 certificate at least contains certificate owner's public key and information about the cert owner.

In the case of web site SSL certificate, digital certificate contains web site's public key and information about the site.

For CA certificates, they contain CA's public key and information about that particular CA.

Digital certificate contains other information as well, like information about issuer, validity, key usages and information that could be used to validate each certificate (e.g. build certification path, perform revocation checking, etc.)

For your question: CA certificate contains its own public key. When bank web site's certificate is validated, a certification path is constructed by using issuer information. Then issuer's public key is used to validate web site certificate's signature. That is, when you validate any particular certificate, public key for signature validation is taken from other source (except self-signed certificates, where public key in the cert is used to validate its own signature).

For more details I would recommend to read RFC5280.