When should we start using stacking of models?
Stacking is going to help most when individual models capture unique characteristics of the data. It is often the case that different architectures perform similarly, if somewhat differently, on the same data. In those cases, ensembling/stacking will only offer slight incremental benefits. In the limit, of you only care about prediction, you can wire up as many different approaches as you can think of. However if interpretability is key, each additional component model will further complicate things. 

Your specific question of when to know if it’s worth it or if you’ve reached the limit can be treated like anything else - is your incremental r-square/error/classification accuracy significantly better versus a simpler approach?