learner supervised learning explanatory variables unsupervised learning supervised and unsupervised learning from note that gt is random function particular outcome is denoted by gt it is often useful to think of teacher-learner metaphorwhereby the function gt is learner who learns the unknown functional relationship gx from the training data we can imagine "teacherwho provides examples of the true relationship between the output yi and the input xi for nand thus "trainsthe learner gt to predict the output of new input xfor which the correct output is not provided by the teacher (is unknownthe above setting is called supervised learningbecause one tries to learn the functional relationship between the feature vector and response in the presence of teacher who provides examples it is common to speak of "explainingor predicting on the basis of xwhere is vector of explanatory variables an example of supervised learning is email spam detection the goal is to train the learner gt to accurately predict whether any future emailas represented by the feature vector xis spam or not the training data consists of the feature vectors of number of different email examples as well as the corresponding labels (spam or not spamfor instancea feature vector could consist of the number of times sales-pitch words like "free""sale"or "miss outoccur within given email as seen from the above discussionmost questions of interest in supervised learning can be answered if we know the conditional pdf ( )because we can then in principle work out the function value (xin contrastunsupervised learning makes no distinction between response and explanatory variablesand the objective is simply to learn the structure of the unknown distribution of the data in other wordswe need to learn (xin this case the guess (xis an approximation of (xand the risk is of the form `(ge lossf ( ) ( ) an example of unsupervised learning is when we wish to analyze the purchasing behaviors of the customers of grocery shop that has total ofsaya hundred items on sale feature vector here could be binary vector { } representing the items bought by customer on visit to the shop ( in the -th position if customer bought item { and otherwisebased on training set { xn }we wish to find any interesting or unusual purchasing patterns in generalit is difficult to know if an unsupervised learner is doing good jobbecause there is no teacher to provide examples of accurate predictions the main methodologies for unsupervised learning include clusteringprincipal component analysisand kernel density estimationwhich will be discussed in in the next three sections we will focus on supervised learning the main supervised learning methodologies are regression and classificationto be discussed in detail in and more advanced supervised learning techniquesincluding reproducing kernel hilbert spacestree methodsand deep learningwill be discussed in and