leave-one-out cross-validation figure shows the cross-validation loss for { the case corresponds to the leave-one-out cross-validationwhich can be computed more efficiently using the formula in theorem model modeling data the first step in any data analysis is to model the data in one form or another for examplein an unsupervised learning setting with data represented by vector [ ] very general model is to assume that is the outcome of random vector [ ]with some unknown pdf the model can then be refined by assuming specific form of when given sequence of such data vectors xn one of the simplest models is to assume that the corresponding random vectors xn are independent and identically distributed (iidwe write iid xn or iid xn distto indicate that the random vectors form an iid sample from sampling pdf or sampling distribution dist this model formalizes the notion that the knowledge about one variable does not provide extra information about another variable the main theoretical use of independent data models is that the joint density of the random vectors xn is simply the product of the marginal onessee theorem specificallyf ,xn ( xn ( (xn in most models of this kindour approximation or model for the sampling distribution is specified up to small number of parameters that isg(xis of the form ( bwhich is known up to some parameter vector examples for the one-dimensional case ( include the (us )bin(np)and exp(ldistributions see tables and for other common sampling distributions typicallythe parameters are unknown and must be estimated from the data in nonparametric setting the whole sampling distribution would be unknown to visualize the underlying sampling distribution from outcomes xn one can use graphical representations such as histogramsdensity plotsand empirical cumulative distribution functionsas discussed in if the order in which the data were collected (or their labelingis not informative or relevantthen the joint pdf of xn satisfies the symmetryf ,xn ( xn xp ,xpn (xp xpn exchangeable ( for any permutation pn of the integers we say that the infinite sequence is exchangeable if this permutational invariance ( holds for any finite subset of the sequence as we shall see in section on bayesian learningit is common to assume that the random vectors xn are subset of an exchangeable sequence and thus satisfy ( note that while iid random variables are exchangeablethe converse is not necessarily true thusthe assumption of an exchangeable sequence of random vectors is weaker than the assumption of iid random vectors