How to detect/handle card failure from Payment Provider's callback URL
I would -


make the return page a CiviCRM IPN callback - civicrm/payment/ipn/XX (where XX is the database ID of the payment processor in use).
from this IPN callback, which is a method in your processor, return a 302 to the Thanks page if the transaction "checks out", and the Unsuccessful page if not


Probably the best "reference" payment processors are NOT those in core, but those based on Omnipay.


Hitting civicrm/payment/ipn/XX will land you in CRM_Core_Payment::handleIPN() which should in turn call your processor's $processorInstance-&gt;$method(); determined via CRM_Core_Payment::handlePaymentMethod().
Eileen's Github probably has a few Omnipay based processors worth checking out.