"""
# Recommendation System for Movielens Dataset - Kaggle
"""
"""
### The Age of Recommender Systems

The rapid growth of data collection has led to a new era of information. Data is being used to create more efficient systems and this is where Recommendation Systems come into play. Recommendation Systems are a type of information filtering systems as they improve the quality of search results and provides items that are more relevant to the search item or are realted to the search history of the user.

They are used to predict the rating or preference that a user would give to an item. Almost every major tech company has applied them in some form or the other: Amazon uses it to suggest products to customers, YouTube uses it to decide which video to play next on autoplay, and Facebook uses it to recommend pages to like and people to follow. Moreover, companies like Netflix and Spotify depend highly on the effectiveness of their recommendation engines for their business and sucees.

#### In this kernel we'll be building a baseline Movie Recommendation System using Movielens Dataset.

"""
"""


### There are basically three types of recommender systems:-

#### Demographic Filtering- 
They offer generalized recommendations to every user, based on movie popularity and/or genre. The System recommends the same movies to users with similar demographic features. Since each user is different , this approach is considered to be too simple. The basic idea behind this system is that movies that are more popular and critically acclaimed will have a higher probability of being liked by the average audience.

#### Content Based Filtering- 
They suggest similar items based on a particular item. This system uses item metadata, such as genre, director, description, actors, etc. for movies, to make these recommendations. The general idea behind these recommender systems is that if a person liked a particular item, he or she will also like an item that is similar to it.

#### Collaborative Filtering- 
This system matches persons with similar interests and provides recommendations based on this matching. Collaborative filters do not require item metadata like its content-based counterparts.



In this notebook we are going to implement content based recommendation system
"""
"""
### Importing essential libraries
"""
import pandas as pd
import matplotlib.pyplot as plt
%matplotlib inline 
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.metrics.pairwise import linear_kernel
from wordcloud import WordCloud
"""
### Loading the Dataset:
"""
"""

The data consists of 105339 ratings applied over 10329 movies.

The movies.csv dataset contains three columns:

    movieId: the ID of the movie
    title: movies title
    genres: movies genres

The ratings.csv dataset contains four columns:

    userId: the ID of the user who rated the movie.
    movieId: the ID of the movie
    ratings: ratings given by each user (from 0 to 5)
    Timstamp: The time the movie was rated.


"""
import os
movie = pd.read_csv('../input/movie-recommendation-engine/movies.csv')
ratings = pd.read_csv('../input/movie-recommendation-engine/ratings.csv')
movie.head()
ratings.head()
movie.info()
print("___________.......................______________")
ratings.info()
# Dimension of datasets:
print("Dimension of Movie Dataset is")
movie.shape
print("Dimension of Ratings Dataset is")
ratings.shape
movie.describe()
ratings.describe()
"""
From the above table we can conclue that in the give dataset:

    The average rating is 3.50 and minimum and maximum rating is 0.5 and 5 respectively.
    There are 668 user who has given their ratings for 149532 movies.


"""
"""
# To generate wordcloud seperate each string for its count into a list:
"""
genres=[]
for genre in movie.genres:
    
    x=genre.split('|')
    for i in x:
         if i not in genres:
            genres.append(str(i))
genres=str(genres)    
movie_title=[]
for title in movie.title:
    movie_title.append(title[0:-7])
movie_title=str(movie_title)    
# Format of both the cloud 
wordcloud_genre=WordCloud(width=1500,height=800,background_color='black',min_font_size=2,min_word_length=3).generate(genres)
wordcloud_title=WordCloud(width=1500,height=800,background_color='cyan',min_font_size=2,min_word_length=3).generate(movie_title)
plt.figure(figsize=(30,15))
plt.axis('off')
plt.title('WORDCLOUD for Movies Genre',fontsize=30)
plt.imshow(wordcloud_genre)
plt.figure(figsize=(30,15))
plt.axis('off')
plt.title('WORDCLOUD for Movies title',fontsize=30)
plt.imshow(wordcloud_title)
"""
## Now we will gonna merge both the dataset:
"""
df = pd.merge(ratings,movie, how='left', on='movieId')
df.head()
"""
### Sum of rating of all the rated movies and top 20 high rated movies table and diagram:
"""
df1=df.groupby(['title'])[['rating']].sum()
df1.head(10)
high_rated=df1.nlargest(20,'rating') # by default first 20 rated movies
high_rated.head()
plt.figure(figsize=(30,10))
plt.bar(high_rated.index,high_rated['rating'])
plt.ylabel('ratings', fontsize=20)
plt.xticks(fontsize=20,rotation=90)
plt.xlabel('Movie Title', fontsize=20)
plt.yticks(fontsize=15)
# after viewing the sum let's check the count of the movies rating how many time movies are rated:
df2=df.groupby('title')[['rating']].count()
rating_count_20=df2.nlargest(20,'rating')
rating_count_20.head()
plt.figure(figsize=(30,10))
plt.title('Top 20 movies with highest number of ratings',fontsize=30)
plt.xticks(fontsize=25,rotation=90)
plt.yticks(fontsize=25)
plt.xlabel('movies title',fontsize=30)
plt.ylabel('Ratings Count',fontsize=30)

plt.bar(rating_count_20.index,rating_count_20.rating,color='red')
"""
# Content Based Filtering¶
"""
"""

Content Based Filtering

In this recommender system the content of the movie (genres, cast, crew, keyword, tagline etc) is used to find its similarity with other movies. Then the movies that are most likely to be similar are recommended.
"""
"""
### Plot description based Recommender
"""
"""
For any of you who has done even a bit of text processing before knows we need to convert the word vector of each overview. Now we'll compute Term Frequency-Inverse Document Frequency (TF-IDF) vectors for each overview.

Now if you are wondering what is term frequency , it is the relative frequency of a word in a document and is given as (term instances/total instances). Inverse Document Frequency is the relative count of documents containing the term is given as log(number of documents/documents with term) The overall importance of each word to the documents in which they appear is equal to TF * IDF

This will give you a matrix where each column represents a word in the overview vocabulary (all the words that appear in at least one document) and each row represents a movie, as before.This is done to reduce the importance of words that occur frequently in plot overviews and therefore, their significance in computing the final similarity score.

Fortunately, scikit-learn gives you a built-in TfIdfVectorizer class that produces the TF-IDF matrix in a couple of lines. That's great, isn't it
"""
#term frequency inverse document frequency:
#Define a TF-IDF Vectorizer Object
CV=TfidfVectorizer()

#Construct the required TF-IDF matrix by fitting and transforming the data
tfidf_matrix=CV.fit_transform(movie['genres'])
#Output the shape of tfidf_matrix
tfidf_matrix.shape
#Creating a pivot table array for our customize view:
movie_user = df.pivot_table(index='userId',columns='title',values='rating')
movie_user.head()
"""
Suppose a user wants to watch a movie similar to burbs, The (1989) then we can recommend the user by calculating the cosine similarity between burbs, The (1989) and other movies. So we have to first find the cosine similarity between the movies of data.

With this matrix in hand, we can now compute a similarity score. There are several candidates for this; such as the euclidean, the Pearson and the cosine similarity scores. There is no right answer to which score is the best. Different scores work well in different scenarios and it is often a good idea to experiment with different metrics.

#### We will be using the cosine similarity to calculate a numeric quantity that denotes the similarity between two movies. We use the cosine similarity score since it is independent of magnitude and is relatively easy and fast to calculate. Mathematically, it is defined as follows:
"""
"""
![image.png](attachment:image.png)
"""
"""
Since we have used the TF-IDF vectorizer, calculating the dot product will directly give us the cosine similarity score. Therefore, we will use sklearn's linear_kernel() instead of cosine_similarities() since it is faster.
"""
# Compute the cosine similarity matrix
cosine_sim = linear_kernel(tfidf_matrix, tfidf_matrix)
"""
### We are going to define a function that takes in a movie title as an input and outputs a list of the 10 most similar movies. Firstly, for this, we need a reverse mapping of movie titles and DataFrame indices. In other words, we need a mechanism to identify the index of a movie in our metadata DataFrame, given its title.
"""
#Construct a reverse map of indices and movie titles
indices=pd.Series(movie.index,index=movie['title']).drop_duplicates()
indices
"""
### We are now in a good position to define our recommendation function. These are the following steps we'll follow :-

    -Get the index of the movie given its title.
    -Get the list of cosine similarity scores for that particular movie with all movies. Convert it into a list of tuples  where the first element is its position and the second is the similarity score.
    -Sort the aforementioned list of tuples based on the similarity scores; that is, the second element.
    -Get the top 10 elements of this list. Ignore the first element as it refers to self (the movie most similar to a particular movie is the movie itself).
    -Return the titles corresponding to the indices of the top elements.


"""
# Function that takes in movie title as input and outputs most similar movies
def get_recommendations(title, cosine_sim=cosine_sim):
    # Get the index of the movie that matches the title
    idx = indices[title]

    # Get the pairwsie similarity scores of all movies with that movie
    sim_scores = list(enumerate(cosine_sim[idx]))

    # Sort the movies based on the similarity scores
    sim_scores = sorted(sim_scores, key=lambda x: x[1], reverse=True)

    # Get the scores of the 10 most similar movies
    sim_scores = sim_scores[1:11]

    # Get the movie indices
    movie_indices = [i[0] for i in sim_scores]

    # Return the top 10 most similar movies
    return movie['title'].iloc[movie_indices]

# without explaination:

#titles=movies['title']
#def recommendations(title):
 #   idx = indices[title]
  #  sim_scores = list(enumerate(cosine_sim[idx]))
   # sim_scores = sorted(sim_scores, key=lambda x: x[1], reverse=True)
   # sim_scores = sim_scores[1:21]
    #movie_indices = [i[0] for i in sim_scores]
    #return titles.iloc[movie_indices]
# # Top 10 Similar movies to Toy Story (1995):
get_recommendations('Toy Story (1995)')
# Top 10 Similar movies to Pulp Fiction (1994):
get_recommendations('Pulp Fiction (1994)')
# Top 10 Similar movies to Jumanji 1995:
get_recommendations('Jumanji (1995)')
get_recommendations('Casino (1995)')
"""
## As you can see, our engine has done a decent job for finding movies with similar plot descriptions.
"""
"""
## Thank You
"""