What is the largest "safe" order for the digital Butterworth filter of a given signal?
One cause is that higher order Butterworth filters have poles closer to the unit circle.  This nearby infinite gain point increases the likelihood of numerical instabilities.  (e.g. rounding/arithmetic/quantization noise may move a pole to the “wrong” side of the unit circle.)
Where the numerical noise will blow up depends on your executable code’s exact arithmetic sequence, and the quantization and number format used.  Often, dividing up a high-order filter into a sequence of biquad stages produces a more stable (better conditioned) sequence of arithmetic operations.