Prediction interval for general non-linear regression I have been working on a predictive model. With each prediction, we need to provide a score to express the confidence about our prediction. So I am looking at prediction interval (PI). In linear regression, I believe these can be obtained and well-documented. However, I am yet to find much reference for non-linear regression (such svr, gbr or other blackbox method for regression). Two methods that I have seen are given below:

1) Using bagging, we can generate many point prediction of each new data point, and then we get the interval from the distribution of these predictions around each new point.

2) using Quantile regression to get the upper and lower bound of the new point. 

Personally, I do like the bagging method, although I don't feel very convinced. Hence, I am reaching out to the community to get a general opinion or some other ideas that I haven't seen so far.
Bootstrapping (which I think you are referring to as bagging, a specific algorithm that incorporates bootstrapping) is your friend here.

Simply train N models using N datasets, where these N datasets are created by sampling from your original dataset with replacement. Using these N different models, generate N predictions on an observation of interest. 

You can now use any sort of method to derive the confidence interval of interest with these N predictions, whether that is using a normality assumption, empirical percentiles (simple bootstrapping), etc.

The one drawback I see with this method is that if you have many observations to predict, this will quickly become computationally expensive. That being said, bootstrapping is easily made parallel.