cannot get low power with Adafruit Trinket
Please have a look at the schematics:
https://cdn-learn.adafruit.com/assets/assets/000/010/774/original/adafruit_products_trinket5.png?1378223478
The USB connector uses a resistor (1500 Ohms) against the 5V net and a 3.6 Volt Z-Diode on it's (D-)-Pin. In addition to the current that is drawn by the microcontroller, this part of the circuit draws some extra current. (5[V] - 3.6[V])/1500[ohm] = 0.0009333... ~= 0.001 [A].
Even though I'm not completely sure without making some tests, that seams plausible to me.
EDIT
(Answer to the OPs comment below.)
The schematics uses a normal diode symbol but has a 3.6 Volt label next to it. I assume that it should be a Zener diode with a reverse voltage drop of 3.6 Volts. If I'm right, the Diode is used to limit the 5 Volts to 3.6 Volts (for what reason ever). The Zener Diode effectively limits  the Voltage over it to 3.6 Volts. The rest of the voltage of 5 Volts must be dropped over the resistor. i.e. 5V - 3.6V = 1.4V. This voltage drop tells us  by Ohms Law that the current through the  resistor is:
I = U / R
I[A] = 1.4[V] / 1500[Ohm]  = (about) 1 mA.

This value is very reasonable for your observations and the current is drawn from the 5V connect, hence you can measure it, even if you would desolder the microcontroller.