How to get a confidence score for predictions?
No matter the model, you can always use the non-parametric bootstrap to construct a confidence interval for any parameter, including predictions (which are actually random variables themselves but are reported as expectations). Here's the general procedure:


Let $N$ denote the number of observations in your training data $X$, and $x_j$ denote the specific observation whose prediction, $\hat{y}_j$, you want a CI for.
Let $K$ denote some number of resampling iterations (Must be $\ge 20$ for a CI with coverage $\ge 95\%$)
For $i$ in $K$, draw a $N$ random samples from $X$ with replacement. Denote this $X_i^{*}$
Train a model on $X_i^{*}$ and use this model to form a prediction on $x_j$. Call this $\hat{y}^{*}_{ji}$
Estimate distributional parameters for $\hat{y}_j$ from your sample. A $100 -
 \alpha$ CI is given by the $\frac{\alpha}{2}$ and $100 - \frac{\alpha}{2}$ percentiles of $\hat{y}^{*}_{j}$.