Programming ATtiny85 with Arduino-IDE and an 8MHz external oscillator
If you look at the Digispark Schematic you'll see resistors and Zener diodes attached to XTAL1 and XTAL2:

These are for the USB D+ and D- signals that the Digispark bit-bangs.  I deeply suspect that the crystal is just not going to resonate with these parts connected.
To get back to a working Digispark you'll probably have to inject signal onto XTAL1 again and reprogram the fuses to internal OSC.