some mllib capabilities sowhat are some of the things mllib can dowellone is feature extraction one thing you can do at scale is term frequency and inverse document frequency stuffand that' useful for creatingfor examplesearch indexes we will actually go through an example of that later in the the keyagainis that it can do this across cluster using massive datasetsso you could make your own search engine for the web with thispotentially it also offers basic statistics functionschi-squared testspearson or spearman correlationand some simpler things like minmaxmeanand variance those aren' terribly exciting in and of themselvesbut what is exciting is that you can actually compute the variance or the mean or whateveror the correlation scoreacross massive datasetand it would actually break that dataset up into various chunks and run that across an entire cluster if necessary soeven if some of these operations aren' terribly interestingwhat' interesting about it is the scale at which it can operate at it can also support things like linear regression and logistic regressionso if you need to fit function to massive set of data and use that for predictionsyou can do that too it also supports support vector machines we're getting into some of the more fancy algorithms heresome of the more advanced stuffand that too can scale up to massive datasets using spark' mllib there is naive bayes classifier built into mllibsoremember that spam classifier that we built earlier in the bookyou could actually do that for an entire -mail system using sparkand scale that up as far as you want to decision treesone of my favorite things in machine learningare also supported by sparkand we'll actually have an example of that later in this we'll also look at -means clusteringand you can do clustering using -means and massive datasets with spark and mllib even principal component analysis and svd (singular value decompositioncan be done with spark as welland we'll have an example of that too andfinallythere' built-in recommendations algorithm called alternating least squares that' built into mllib personallyi've had kind of mixed results with ityou knowit' little bit too much of black box for my tastebut am recommender system snobso take that with grain of saltspecial mllib data types using mllib is usually pretty straightforwardthere are just some library functions you need to call it does introduce few new data typeshoweverthat you need to know aboutand one is the vector