MySQL is using wrong index
MySQL will chose the index based on few factors. One of them is the cardinality of the indexed field, and other one in the number of rows to be scanned (on average). In your first example, MySQL decided that it will read less rows if it is used the index on "answer_id". 
If you provides the stats of your table, it will be much clearer. 
Example of data snapshot when this happens:

user_id answer_id order
48      1         15   
48      2         17
48      3         19
48      4         13


Here, if MySQL will see that user (User_id, answer_id) will scan 4 rows, while using (exam_id) will scan only 1 row.