ATtiny2313 refuses to be programmed after setting CPU clock to 500kHz
The issue, as explained, too high SPI clock frequency. There exists a fixed version of ArduinoISP that can program at lower frequency available here. Some discussion on topic is available here.