What is meant by 'stability'?
Stability has several meanings. A compound can be stable on its own, under the given conditions such as temperature and pressure. This is the case for a lot of compounds. The other definition of stable could be a compound's inability to react with other molecules (like the inability of carboxylic acids to be oxidized). This is probably what you meant. A good base, like $R-O^-$ is unstable because of the +I effect of the alkyl group, which shifts the electron density towards oxygen, which in turn makes the ion a very good and strong nucleophile. This is the reason why such "good bases" tend to cause eliminations  and substitutions in organic molecules. 

P.S.: This is the simplest explanation I can provide. If you have any doubts, ask them in the comments.

P.S.S.: Stability of a compound can also be explained on the basis of what @NotNicolaou said.