probability and statistics in generalthe data used to fit the model is viewed as realization of random process or numerical vectorwhose probability law determines the accuracy with which we can predict future observations thusin order to quantify the uncertainty inherent in making predictions about the futureand the sources of error in the modeldata scientists need firm grasp of probability theory and statistical inference feature response prediction function regression classification loss function supervised and unsupervised learning given an input or feature vector xone of the main goals of machine learning is to predict an output or response variable for examplex could be digitized signature and binary variable that indicates whether the signature is genuine or false another example is where represents the weight and smoking habits of an expecting mother and the birth weight of the baby the data science attempt at this prediction is encoded in mathematical function gcalled the prediction functionwhich takes as an input and outputs guess (xfor (denoted by yfor examplein senseg encompasses all the information about the relationship between the variables and yexcluding the effects of chance and randomness in nature in regression problemsthe response variable can take any real value in contrastwhen can only lie in finite setsay { }then predicting is conceptually the same as classifying the input into one of categoriesand so prediction becomes classification problem we can measure the accuracy of prediction with respect to given response by using some loss function loss( , yin regression setting the usual choice is the squared error loss ( - yin the case of classificationthe zero-one (also written - loss function loss( , { yis often usedwhich incurs loss of whenever the predicted class is not equal to the class later on in this bookwe will encounter various other useful loss functionssuch as the cross-entropy and hinge loss functions (seee the word error is often used as measure of distance between "trueobject and some approximation thereof if is real-valuedthe absolute error | yand the squared error ( - yare both well-established error conceptsas are the norm ky- yk and squared norm ky - yk for vectors the squared error ( - yis just one example of loss function risk it is unlikely that any mathematical function will be able to make accurate predictions for all possible pairs (xyone may encounter in nature one reason for this is thateven with the same input xthe output may be differentdepending on chance circumstances or randomness for this reasonwe adopt probabilistic approach and assume that each pair (xyis the outcome of random pair (xythat has some joint probability density (xywe then assess the predictive performance via the expected lossusually called the riskfor `(ge loss(yg( )( for examplein the classification case with zero-one loss function the risk is equal to the probability of incorrect classification`(gp[ ( )in this contextthe prediction