Cryptography used for login and signing transactions My bank uses a smartcard reader (separate module not connected to the computer) as depicted below:


When I need to log into my account it provides me with a challenge and  I need to provide some details:



I need to provide the 8 digits, enter my pin and then it returns an 8 number sequence which is then to be entered after which you are logged in.

A similar mechanism is employed for signing transactions but using the M2 button (blue one) rather than the M1 and instead of a random number you need to first give the amount and then the timestamp. I'm interested in the cryptography used behind it (if it's symmetric, assymetric or a combination of both) and how the signing is actually done. 

a user guide can be found here.
While I am not familiar with this particular device or bank, this looks a lot like Chip Authentication Program (CAP)

It's fairly standard in the UK, where banks use this to authenticate initiating a bank transfer to a new recipient. There is a python implementation (for study - it's obviously a bad idea to actually use this, as it would permit your PIN to be captured by malware on your local PC) available at http://sites.uclouvain.be/EMV-CAP/Application/

As I understand the protocol, your smartcard is used to sign a "cancel transaction" request using a shared MAC key (the key being securely stored on the smartcard). There is more information on the protocol at the wikipedia page: https://en.wikipedia.org/wiki/Chip_Authentication_Program#Protocol_details