What encoding or format is the private key in Dumpprivkey? I'm trying to use the results of Dumpprivkey to sign a message outside of the QT client, however I'm either having a problem signing or extracting the real value of the key.

What encoding, endian, or format is Dumpprivkey in?  

I assume it's Base58 encoded...
The format is base58check, with 0x80 as version byte (for Bitcoin mainnet, at least), and an optional suffix 0x01 byte to indicate that the corresponding public key is compressed. So combined:


base58(0x80 | 32-byte big-endian secret parameter | checksum) for uncompressed public keys
base58(0x80 | 32-byte big-endian secret parameter | 0x01 | checksum) for compressed public keys