The Scot, 30, was set to be in the British line-up to defend the title in Brazil but has been ruled out after his top two rides picked up injuries.
Hello Sanctos and Hello M'Lady picked up minor injuries at the start of the season and will not be fit for Rio.
Brash was the top-ranked showjumper in the world last year and is still GB's leading rider in the global standings.
Scott Brash, part of Team GB's Olympic gold-medal winning showjumping team at London 2012, will miss the Rio Games.