case study predicting malicious urls listing creating model to distinguish the malicious from the normal urls we know number of features from data exploration the target variable can be or - " "website safe to visit"- "website unsafe set up stochastic gradient classifier classes [- , sgd sgdclassifier(loss="log"n_features= split all files together take up around gb the trick here is to leave data for tarinfo in tarcompressed in main memory and if splitonly unpack what you need break if tarinfo isfile() tar extractfile(tarinfo namex, load_svmlight_file( ,n_features=n_featuresif splitsgd partial_fit(xyclasses=classesif =splitprint classification_report(sgd predict( ),yi + stop when we reach files and print results initialize file counter at we unpack the files one by one to reduce the memory needed use helper functionload_svmlight_file(to load specific file third important thing is online algorithm it can be fed data points file by file (batchesstop at th file (instead of all of themfor demonstration purposesthe code in the previous listing looks fairly similar to what we did beforeapart from the stochastic gradient descent classifier sgdclassifier(herewe trained the algorithm iteratively by presenting the observations in one file with the partial_fit(function looping through only the first files here gives the output shown in table the table shows classification diagnostic measuresprecisionrecallf -scoreand support table classification problemcan website be trusted or notprecision recall -score support - avg/total only ( of the malicious sites aren' detected (precision)and ( of the sites detected are falsely accused (recallthis is decent resultso we can conclude that the methodology works if we rerun the analysisthe result might be slightly