Why BiCl5 isn't stable?
The reason behind this is the mainly inert pair effect . 

In $\ce{BiCl3}$, due to the much electronegativity difference between $\ce{Bi}$ and $\ce{Cl}$, Chlorine atom forms bonds with almost pure $p$ orbitals of $\ce{Bi}$, and the lone pairs on $\ce{Bi}$ are of almost pure $s$ character. Thus, $\ce{Bi}$ atom doesn't actually utilise much of its $s$ orbital electrons in forming the bonds which is energetically much more preferable. ( If it seems non-obvious, recall Bent's rule which says that more electronegative atoms prefer to form bonds with orbitals with more $p$ - character.)

On the other hand, if $\ce{Bi}$ is present in $\ce{BiCl5}$, it has no other option other than utilising its $s$ orbitals for bonding. The equatorial bonds will mainly consist $s$ character and $p_x, p_y$ orbitals will also participate in forming the equatorial bonds. So, $s$ character in the equatorial bonds will be slightly lesser than $33$% (again due to Bent's rule). And the axial bonds will consist of mainly $p_z $ and $d_{z^2}$ orbitals and little bit of $s$ mixing may also be there as it is spherically symmetric. So, overall central atom $\ce{Bi}$ has to use its $s$ electrons in bonding which is difficult energetically.

Now, the participation of $s$ electrons is difficult in bonding is due to the relativistic contraction of $s$ orbitals. It becomes closer to the nucleus and more stable in case of heavy elements like $\ce{Au, Hg, Tl, Pb, Bi}$ etc. Hence, the central atom needs to pay more energy to involve those stable electrons in bonding. That's why, the $s$ electrons become kind of inert and thus these heavy elements don't prefer higher oxidation states to avoid participation of these inner $s$ orbitals in bonding.