What's special about "unity-gain stable"?
Stability doesn't only depend on gain, but also phase. If an inverting amplifier has a 180° phase shift total phase shift is 360°, and one of the Barkhausen criteria for oscillation is met.


  Amplifiers differ in their ability to be stable even if the external
  circuitry is optimum. To evaluate the stability potential for a
  particular amplifier type, graphic data is required for both "gain vs
  frequency" and "phase vs frequency" of the open loop amplifier. If
  the phase response exhibits !180E at a frequency where the gain is
  above unity, the negative feedback will become positive feedback
  and the amplifier will actually sustain an oscillation. Even if the
  phase lag is less than !180E and there is no sustained oscillation,
  there will be overshoot and the possibility of oscillation bursts
  triggered by external noise sources, if the phase response is not
  "sufficiently less" than -180° for all frequencies where the gain is
  above unity. The "sufficiently less" term is more properly called
  phase margin. If the phase response is -135°, then the phase
  margin is 45° (the amount "less than" -180°). Actually, the phase
  margin of interest to evaluate stability potential must also include the
  phase response of the feedback circuit. When this combined phase
  margin is 45° or more, the amplifier is quite stable. The 45° number
  is a "rule of thumb" value and greater phase margin will yield even
  better stability and less overshoot.
  
  Often, but not always, the lowest phase margin is at the highest
  frequency which has gain above unity; because there is always some
  delay independent of frequency which represents more degrees at
  higher frequencies. An amplifier with 45E phase margin at the higher
  frequency of unity open loop gain is said to be "unity gain stable".
  Optionally, most amplifier types can be compensated for unity gain
  stability at some sacrifice in slew rate or high frequency noise. If
  stability is considered to be of high priority, the tradeoff must be
  made. Unity gain stable means stable operation at the lowest closed
  loop gain where stability is usually worst.


(from here)

Further reading
Why Unity Feedback is Most Difficult for Stability?