Are there any poisons with an activation threshold?
Yes, there are certainly some chemicals that are harmless in low quantities but really dangerous in higher doses.

The common painkiller paracetamol (or acetaminophen) is one such candidate. In low doses it is a mild analgesic but in higher does it is deadly. The reason is that there is a metabolic side-reaction in the liver that produces a small amount of a toxic compound. Under normal circumstances this is mopped up by other pathways in the liver and causes no damage. But when large amounts are taken the liver's capacity to mop up the toxin is exhausted, the toxin builds up to dangerous levels and it causes severe liver damage often resulting in death.