SD test is taking long answer when it fails
The SD library builds on the sdfatlib library, in which SD_INIT_TIMEOUT is defined as 2000 ms, which gives your 2 second timeout. You could try lowering that number to get a faster response, although that may run the risk of timing out even when a card is present.  

Four different timeout periods are defined in the sdfatlib source:

/** init timeout ms */
uint16_t const SD_INIT_TIMEOUT = 2000;
/** erase timeout ms */
uint16_t const SD_ERASE_TIMEOUT = 10000;
/** read timeout ms */
uint16_t const SD_READ_TIMEOUT = 300;
/** write time out ms */
uint16_t const SD_WRITE_TIMEOUT = 600;


The 300 ms read timeout is faster, as @Paul suggests in his comment, so you could just use a card read as the test rather than an initialisation, or try setting the init timeout to 300 instead. A read test would be simpler, as it doesn't involve modifying library files. It might not cope with a re-inserted card, though - I suspect that requires reinitialisation each time.

Note: the version of sdfatlib included with the SDlib in my Arduino install appears rather older than the current version. The included version defines the timeouts in Sd2Card.h, while the latest has is in SdInfo.h. The snippet above is from the older, included code.