Difference between output of probabilistic and ordinary least squares regressions
They both solve the exact same objective, which is minimizing the mean squared error. However the second method can answer "how confident it is that slope is not zero, i.e. $Y$ is correlated with $X$?" via p-value.

In detail

Lets denote the data as $(X, Y) = \{(x_n, y_n)|x_n \in \mathbb{R}^D, y_n \in \mathbb{R}\}$. And the regression as $\hat{y} = Ax+B$.

Extra quantities returned by scipy.stats.linregress(X,Y) are: rvalue ($r$), and pvalue ($p$). 

In statistics, $r^2$ (known as r-squared) measures the "goodness-of-fit" . That is, as regression $\hat{y}=Ax+B$ gets closer to observation $y$, $r^2$ gets closer to $1$. Since it is a function of $y$ and $\hat{y}$, it can be calculated for the first method too. So no difference here.

However, $p$ is specific to second method. scipy.stats.linregress(X,Y) adds a normality assumption to noise, i.e. assumes $\epsilon \sim  N(0, \sigma^2)$ where $$\epsilon = y - \overbrace{Ax+B}^{\hat{y}}$$
On the basis of this assumption, it can answer an additional question: "how confident it is that the slope is not zero?". The first method cannot answer this question. 

For example, suppose the estimated slope is $2.1$ for both methods, we still cannot tell whether this slope is significant or $Y$ is actually independent of $X$. Unless we look at the value of $p$. For example, for $p &lt; 0.01$ we are confident (at significance level $0.01$) that $Y$ is correlated with $X$, but for $p &gt; 0.1$ we cannot be confident, i.e. slope $2.1$ could be due to chance and $Y$ might be independent of $X$.

This link gives more details on how p-value is actually calculated in second method.