23 Web Science/Part2: Emerging Web Properties Web Science/Part2: Emerging Web Properties/MoocIndex Web Science/Part2: Emerging Web Properties/Modelling Similarity of Text/Comparing Results of Similarity Merasures edit MOOC index Learning goals Video Script Quiz Further reading Discussion Understand that different modeling choices can produce very different results. Have a feeling how you could statistically compare the differences of the models. Know how you could extract keywords from documents with the tf-idf approach. Try to argue which model you like best in a certain scenario. The script can be Found at File:Comparing-Results-of-Similarity-Merasures.pdf 1 which method can be used best to find characteristic words of a text? 2 Which method works well in an information retrieval setting 3 Which method should be used when you don't have several occurences of the same elements? tba Navigation How big is the World Wide Web Problems with the question about the size of the Web 3 ways to study the Web A simplistic descriptive model An unrealistic, simplistic generative model Summary, further reading, homework Simple statistical descriptive Models for the Web Counting Words And Documents Typical length of a document How to formulate a research hypothesis Number of words needed to understand most of Wikipedia Linguists way of checking simplicity of text Advanced statistical descriptive models for the Web The Zipf law for text Visually straight lines on log log plots Fitting a curve on a log log plot Zipf law powerlaw or pareto law.webm Modelling Similarity of Text Similarity Measures and their Applications Jaccard Similarity for Sets Cosine Similarity For Vectorspaces Probabilistic Similarity Measures Kullback Leibler Divergence Comparing Results of Similarity Merasures Generative Models for the Web Introduction to generative modelling.webm Sampling from a probability distribution Evaluating a generative model Pittfalls when increasing the number of model parameters Modeling the Web as a graph Reviewing terms from graph theory The standard web graph model Descriptive statistics of the web graph Topology of the web graph Modelling-graphs-with-linear-algebra Play media Probabilistic Similarity Measures Kullback Leibler Divergence
{'wiki_source': 'wikiversity', 'other_info': "{'content_model': 'wikitext', 'language': 'en', 'title': 'Web Science/Part2: Emerging Web Properties/Modelling Similarity of Text/Comparing Results of Similarity Merasures', 'type': 'text'}"}