Is there any formal explanation for the sensitivity of AdaBoost to outliers?
Consider a supervised problem with target values y either being -1 or 1. Let's say we are fitting function f(x) using our AdaBoost classifier. Intuitively we want positive margins, where a margin is defined as $y_{i}f(x_{i})$. A positive margin means we have correctly classified that sample since the sign of both f and y are the same. A negative margin means we have misclassified the sample.

The loss function for Adaboost is defined as an exponential:

$L(y, f(x)) = exp(-yf(x))$

This exponential then ends up more strongly penalizing points with a large negative margin (i.e., outliers) than other classifiers. Here is a plot of the loss function vs. the margin for several classifiers:



So, to summarize: Adaboost is susceptible to overfitting outliers due to the form of its loss function.

Source: Hastie, et. al, The Elements of Statistical Learning, 2017, pp. 343-347