% Generated by roxygen2: do not edit by hand
% Please edit documentation in R/roo.R
\name{conformal.pred.roo}
\alias{conformal.pred.roo}
\title{In-sample split conformal prediction intervals.}
\usage{
conformal.pred.roo(
  x,
  y,
  train.fun,
  predict.fun,
  alpha = 0.1,
  mad.train.fun = NULL,
  mad.predict.fun = NULL,
  split = NULL,
  seed = NULL,
  verbose = FALSE
)
}
\arguments{
\item{x}{Matrix of features, of dimension (say) n x p.}

\item{y}{Vector of responses, of length (say) n.}

\item{train.fun}{A function to perform model training, i.e., to produce an
estimator of E(Y|X), the conditional expectation of the response variable
Y given features X. Its input arguments should be x: matrix of features,
and y: vector of responses.}

\item{predict.fun}{A function to perform prediction for the (mean of the)
responses at new feature values. Its input arguments should be out: output
produced by train.fun, and newx: feature values at which we want to make
predictions.}

\item{alpha}{Miscoverage level for the prediction intervals, i.e., intervals
with coverage 1-alpha are formed. Default for alpha is 0.1.}

\item{mad.train.fun}{A function to perform training on the absolute residuals
i.e., to produce an estimator of E(R|X) where R is the absolute residual
R = |Y - m(X)|, and m denotes the estimator produced by train.fun.
This is used to scale the conformal score, to produce a prediction interval
with varying local width. The input arguments to mad.train.fun should be
x: matrix of features, and y: vector of absolute residuals. The default for
mad.train.fun is NULL, which means that no training is done on the absolute
residuals, and the usual (unscaled) conformal score is used. Note that if
mad.train.fun is non-NULL, then so must be mad.predict.fun (see next).}

\item{mad.predict.fun}{A function to perform prediction for the (mean of the)
absolute residuals at new feature values. Its input arguments should be
out: output produced by mad.train.fun, and newx: feature values at which we
want to make predictions. The default for mad.predict.fun is NULL, which
means that no local scaling is done for the conformal score, i.e., the
usual (unscaled) conformal score is used.}

\item{split}{Indices that define the data-split to be used (i.e., the indices
define the first half of the data-split). Default is NULL, in which case
the split is chosen randomly.}

\item{seed}{Integer to be passed to set.seed before defining the random 
data-split to be used. Default is NULL, which effectively sets no seed.
If both split and seed are passed, the former takes priority and the latter
is ignored.}

\item{verbose}{Should intermediate progress be printed out? Default is FALSE.}
}
\value{
A list with the following components: pred, lo, up, fit, split,
  fit.all, out.all.
  The first four are matrices of dimension n x m. In a sense, each
  of the m columns really corresponds to a different prediction function; 
  see details below. Hence, the rows of the matrices pred, lo, up give the
  predicted value, and lower and upper confidence limits (from rank-one-out
  split conformal inference), respectively, for the response at the n points
  given in x. The rows of the matrix fit give the fitted values for the n
  points given in x. The indices used for the half of the data-split are
  returned in split. Finally, for convenience, the output from running
  train.out on the entire (unsplit) data set x,y is stored in out.all, and
  fit.all is an n x m matrix whose rows store the fitted values from out.all.
}
\description{
Compute prediction intervals, having valid in-sample coverage, using
  rank-one-out (ROO) split conformal inference.
}
\details{
For concreteness, suppose that we want to use the predictions from
  forward stepwise regression at steps 1 through 5 in the path. In this case,
  there are m = 5 internal tuning parameter values to predict.fun, in the
  notation used above, and each of the returned matrices pred, lo, up, fit
  will have 5 columns (one for each step of the forward stepwise path).
  The code is structured in this way so that we may defined a single pair of
  functions train.fun and predict.fun, over a set of m = 5 tuning parameter
  values, instead of calling the conformal function separately m = 5 times.

  The rank-one-out or ROO variant of split conformal prediction produces
  intervals with a special in-sample average coverage property. Namely, if
  we were to observe new
  response values y* at the given feature values x, then the ROO method
  produces a prediction band \eqn{\hat{C}(x)} with the asymptotic in-sample
  average coverage property
  \deqn{\liminf_{n \to \infty} \frac{1}{n} \sum_{i=1}^n 
  P \big(Y_i^* \in \hat{C}(X_i) \big) \geq 1-\alpha.}
  The usual split conformal band would not necessarily share this in-sample
  property. Of course, the split conformal band has valid predictive (i.e.,
  out-of-sample) coverage, and the ROO split conformal variant maintains
  this property as well.
}
\examples{
## Lasso: use a fixed sequence of 100 lambda values

# Generate some example training data
set.seed(33)
n = 200; p = 500; s = 10
x = matrix(rnorm(n*p),n,p)
beta = c(rnorm(s),rep(0,p-s)) 
y = x \%*\% beta + rnorm(n)

# Generate some example test data, but using the same x points
n0 = n
x0 = x
y0 = x0 \%*\% beta + rnorm(n0)

# Grab a fixed lambda sequence from one call to glmnet, then
# define lasso training and prediction functions
if (!require("glmnet",quietly=TRUE)) {
  stop("Package glmnet not installed (required for this example)!") 
}
out.gnet = glmnet(x,y,nlambda=100,lambda.min.ratio=1e-4)
lambda = out.gnet$lambda
funs = lasso.funs(lambda=lambda)

# Rank-one-out conformal inference, and split conformal inference
out.roo = conformal.pred.roo(x, y, alpha=0.1,
  train.fun=funs$train, predict.fun=funs$predict)

out.split = conformal.pred.split(x, y, x0, alpha=0.1,
  train.fun=funs$train, predict.fun=funs$predict)

y0.mat = matrix(rep(y0,ncol(out.roo$lo)),nrow=n0)
cov.roo = colMeans(out.roo$lo <= y0.mat & y0.mat <= out.roo$up)
len.roo = colMeans(out.roo$up - out.roo$lo)
err.roo = colMeans((y0.mat - out.roo$pred)^2)

cov.split = colMeans(out.split$lo <= y0.mat & y0.mat <= out.split$up)
len.split = colMeans(out.split$up - out.split$lo)
err.split = colMeans((y0.mat - out.split$pred)^2)

# Compare to parametric intervals from oracle linear regression
lm.orac = lm(y~x[,1:s])
out.orac = predict(lm.orac,newdata=list(x=x0[,1:s]),
  interval="predict", level=0.9)

cov.orac = mean(out.orac[,"lwr"] <= y0 & y0 <= out.orac[,"upr"])
len.orac = mean(out.orac[,"upr"] - out.orac[,"lwr"])
err.orac = mean((y0 - out.orac[,"fit"])^2)
  
# Plot average coverage 
plot(log(lambda),cov.roo,type="o",pch=20,ylim=c(0,1),
     xlab="log(lambda)",ylab="Avg coverage",
     main=paste0("ROO conformal + lasso (fixed lambda sequence):",
       "\nAverage coverage"))
points(log(lambda),cov.split,type="o",pch=20,col=4)
abline(h=cov.orac,lty=2,col=2)
legend("bottomleft",col=c(1,4,2),lty=c(1,1,2),
       legend=c("ROO conformal","Split conformal","Oracle"))

# Plot average length
plot(log(lambda),len.roo,type="o",pch=20,
     ylim=range(len.roo,len.split,len.orac),
     xlab="log(lambda)",ylab="Avg length",
     main=paste0("Split conformal + lasso (fixed lambda sequence):",
       "\nAverage length"))
points(log(lambda),len.split,type="o",pch=20,col=4)
abline(h=len.orac,lty=2,col=2)
legend("bottomleft",col=c(1,4,2),lty=c(1,1,2),
       legend=c("ROO conformal","Split conformal","Oracle"))

# Plot test error
plot(log(lambda),err.roo,type="o",pch=20,
     ylim=range(err.roo,err.split,err.orac),
     xlab="log(lambda)",ylab="Test error",
     main=paste0("Split conformal + lasso (fixed lambda sequence):",
       "\nTest error"))
points(log(lambda),err.split,type="o",pch=20,col=4)
abline(h=err.orac,lty=2,col=2)
legend("bottomleft",col=c(1,4,2),lty=c(1,1,2),
       legend=c("ROO conformal","Split conformal","Oracle"))
}
\references{
"Distribution-Free Predictive Inference for Regression" by Lei,
  G'Sell, Rinaldo, Tibshirani, Wasserman (2018).
}
\seealso{
\code{\link{conformal.pred}}, 
  \code{\link{conformal.pred.jack}}, \code{\link{conformal.pred.split}}
}
