PKCS#10 why keeping the Subject Public Key in the X.509 Certificate
The certificate is signed with the Certificate Authority's private key, but the purpose of an X.509 Public Key certificate is to associate a public key with a subject-entity.


You generate a public/private pair
You generate a PKCS#10 CSR which contains the SubjectPublicKeyInfo, signed with the private key as proof-of-possession.
You submit the PKCS#10 CSR to a CA.
The CA does whatever validation they do to ensure that you're the subject that you claim to be in the CSR (or whatever subject you claim to be external to the CSR) ... and for whatever extensions you may have requested.
The CA copies the SubjectPublicKeyInfo into a new certificate they create, and they sign it with their private key.
People can now use the trust of the certificate to understand that signatures that come "from you" and verify under the public key within the certificate are actually from you (or used a stolen/cracked version of your private key).


Alternatively: Whose public key do you want them to put in your certificate?  If they just made one up on the fly for you they'd also have had a copy of your private key, which is clearly less secure than only you ever having your private key.