Commit
·
77c8b76
1
Parent(s):
bea99bf
Updated readme
Browse files- .gitattributes +1 -1
- .gitignore +1 -2
- README.md +21 -4
- assets/animes.jpg +0 -0
- assets/artifacts.png +0 -0
- assets/collaborative_and_contentbased_filtering.png +0 -0
- logs/02_03_2025_11_40_31.log +129 -0
- run_pipeline.py +61 -46
.gitattributes
CHANGED
|
@@ -32,4 +32,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 32 |
*.xz filter=lfs diff=lfs merge=lfs -text
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
-
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
| 32 |
*.xz filter=lfs diff=lfs merge=lfs -text
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
+
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
.gitignore
CHANGED
|
@@ -1,5 +1,4 @@
|
|
| 1 |
anime/
|
| 2 |
.env
|
| 3 |
-
Artifacts/
|
| 4 |
-
logs/
|
| 5 |
__pycache__/
|
|
|
|
| 1 |
anime/
|
| 2 |
.env
|
| 3 |
+
Artifacts/
|
|
|
|
| 4 |
__pycache__/
|
README.md
CHANGED
|
@@ -12,12 +12,17 @@ license: apache-2.0
|
|
| 12 |
|
| 13 |
Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
|
| 14 |
|
| 15 |
-
#
|
| 16 |
|
| 17 |
-
This is an **Anime Recommendation System** that combines multiple recommendation techniques such as **Collaborative Filtering**, **Content-Based Filtering**, and **Popularity-Based Filtering**. The system is designed
|
| 18 |
|
| 19 |
The system is hosted on **Hugging Face Spaces** and fetches datasets and pre-trained models from **Hugging Face Hub** to generate **personalized anime recommendations** based on user preferences and anime features.
|
| 20 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 21 |
## Tech Stacks 🛠️
|
| 22 |
|
| 23 |
- **Python**: Main programming language used for building recommendation algorithms and Streamlit app.
|
|
@@ -25,7 +30,7 @@ The system is hosted on **Hugging Face Spaces** and fetches datasets and pre-tra
|
|
| 25 |
- **Streamlit**: For building and deploying the web app that serves the recommendations.
|
| 26 |
- **Hugging Face Spaces**: Hosts the Streamlit-based recommendation system.
|
| 27 |
- **Hugging Face Datasets**: Stores and retrieves anime datasets for processing.
|
| 28 |
-
- **Hugging Face Models**:
|
| 29 |
|
| 30 |
## Pipeline 🚀
|
| 31 |
|
|
@@ -47,6 +52,8 @@ The pipeline follows a structured sequence of steps to build an **Anime Recommen
|
|
| 47 |
- The chosen model is trained using **transformed data**, and the final trained model is stored as an artifact.
|
| 48 |
- Once trained, it can generate recommendations for users or anime titles.
|
| 49 |
|
|
|
|
|
|
|
| 50 |
### 4. Content-Based Filtering 🎭
|
| 51 |
- Uses extracted anime features like genres to train a **Content-Based Recommendation Model**.
|
| 52 |
- This model recommends anime similar to those a user has watched or liked.
|
|
@@ -70,13 +77,15 @@ All intermediate and final outputs, including processed datasets and trained mod
|
|
| 70 |
|
| 71 |
When building the **Streamlit** app, these datasets and trained models are retrieved directly from **Hugging Face**, ensuring seamless integration and scalability.
|
| 72 |
|
|
|
|
|
|
|
| 73 |
- The datasets used in this project are available at:
|
| 74 |
- [Anime and User Ratings](https://www.kaggle.com/datasets/krishnaveniponna/anime-and-ratings-list-dataset-2023)
|
| 75 |
|
| 76 |
- You can find the Artifacts of trained models here:
|
| 77 |
- [Pre-trained Models](https://huggingface.co/krishnaveni76/anime-recommendation-models)
|
| 78 |
|
| 79 |
-
##
|
| 80 |
|
| 81 |
This project is deployed using **Hugging Face Spaces**, which provides a seamless way to host **Streamlit applications**. The application pulls the datasets and trained models from Hugging Face and runs inside a **Docker container**.
|
| 82 |
|
|
@@ -111,6 +120,14 @@ source venv/bin/activate
|
|
| 111 |
pip install -r requirements.txt
|
| 112 |
```
|
| 113 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 114 |
### Running with Docker 🚀
|
| 115 |
To run the application inside a Docker container, follow these steps:
|
| 116 |
|
|
|
|
| 12 |
|
| 13 |
Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
|
| 14 |
|
| 15 |
+
# Anime Recommendation system
|
| 16 |
|
| 17 |
+
This is an **Anime Recommendation System** that combines multiple recommendation techniques such as **Collaborative Filtering**, **Content-Based Filtering**, and **Popularity-Based Filtering**. We used the **AnimeList 2023 dataset**, but to optimize computational cost and storage, we included only animes with an average rating above **6.0**. The system is designed for **continuous data ingestion and transformation** and is fully **dockerized** for seamless deployment.
|
| 18 |
|
| 19 |
The system is hosted on **Hugging Face Spaces** and fetches datasets and pre-trained models from **Hugging Face Hub** to generate **personalized anime recommendations** based on user preferences and anime features.
|
| 20 |
|
| 21 |
+
|
| 22 |
+
|
| 23 |
+
## Live Demo 🤗
|
| 24 |
+
[Anime Recommendation System App](https://huggingface.co/spaces/krishnaveni76/Anime-Recommendation-System)
|
| 25 |
+
|
| 26 |
## Tech Stacks 🛠️
|
| 27 |
|
| 28 |
- **Python**: Main programming language used for building recommendation algorithms and Streamlit app.
|
|
|
|
| 30 |
- **Streamlit**: For building and deploying the web app that serves the recommendations.
|
| 31 |
- **Hugging Face Spaces**: Hosts the Streamlit-based recommendation system.
|
| 32 |
- **Hugging Face Datasets**: Stores and retrieves anime datasets for processing.
|
| 33 |
+
- **Hugging Face Models**: Stores the pre-trained recommendation models for inference.
|
| 34 |
|
| 35 |
## Pipeline 🚀
|
| 36 |
|
|
|
|
| 52 |
- The chosen model is trained using **transformed data**, and the final trained model is stored as an artifact.
|
| 53 |
- Once trained, it can generate recommendations for users or anime titles.
|
| 54 |
|
| 55 |
+
|
| 56 |
+
|
| 57 |
### 4. Content-Based Filtering 🎭
|
| 58 |
- Uses extracted anime features like genres to train a **Content-Based Recommendation Model**.
|
| 59 |
- This model recommends anime similar to those a user has watched or liked.
|
|
|
|
| 77 |
|
| 78 |
When building the **Streamlit** app, these datasets and trained models are retrieved directly from **Hugging Face**, ensuring seamless integration and scalability.
|
| 79 |
|
| 80 |
+
|
| 81 |
+
|
| 82 |
- The datasets used in this project are available at:
|
| 83 |
- [Anime and User Ratings](https://www.kaggle.com/datasets/krishnaveniponna/anime-and-ratings-list-dataset-2023)
|
| 84 |
|
| 85 |
- You can find the Artifacts of trained models here:
|
| 86 |
- [Pre-trained Models](https://huggingface.co/krishnaveni76/anime-recommendation-models)
|
| 87 |
|
| 88 |
+
## Deployment on Hugging Face Spaces 🚀
|
| 89 |
|
| 90 |
This project is deployed using **Hugging Face Spaces**, which provides a seamless way to host **Streamlit applications**. The application pulls the datasets and trained models from Hugging Face and runs inside a **Docker container**.
|
| 91 |
|
|
|
|
| 120 |
pip install -r requirements.txt
|
| 121 |
```
|
| 122 |
|
| 123 |
+
### Running the Pipeline 🔄
|
| 124 |
+
To process the data and train the recommendation models, run the following command:
|
| 125 |
+
|
| 126 |
+
```bash
|
| 127 |
+
python run_pipeline.py
|
| 128 |
+
```
|
| 129 |
+
This will execute the pipeline, ingest and transform data, and train the models before making recommendations.
|
| 130 |
+
|
| 131 |
### Running with Docker 🚀
|
| 132 |
To run the application inside a Docker container, follow these steps:
|
| 133 |
|
assets/animes.jpg
ADDED
|
assets/artifacts.png
ADDED
|
assets/collaborative_and_contentbased_filtering.png
ADDED
|
logs/02_03_2025_11_40_31.log
ADDED
|
@@ -0,0 +1,129 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
[ 2025-02-03 11:40:38,249 ] 8 root - INFO - Starting the Anime Recommendation System Training Pipeline...
|
| 2 |
+
[ 2025-02-03 11:40:38,249 ] 42 root - INFO - Initiating Data Ingestion...
|
| 3 |
+
[ 2025-02-03 11:40:38,249 ] 40 root - INFO - Fetching data from Hugging Face dataset: krishnaveni76/Animes
|
| 4 |
+
[ 2025-02-03 11:40:50,688 ] 48 root - INFO - Shape of the dataframe: (12194, 18)
|
| 5 |
+
[ 2025-02-03 11:40:50,689 ] 49 root - INFO - Column names: Index(['anime_id', 'genres', 'name', 'average_rating', 'overview', 'type',
|
| 6 |
+
'episodes', 'producers', 'licensors', 'studios', 'source',
|
| 7 |
+
'anime_rating', 'rank', 'popularity', 'favorites', 'scored by',
|
| 8 |
+
'members', 'image url'],
|
| 9 |
+
dtype='object')
|
| 10 |
+
[ 2025-02-03 11:40:50,718 ] 50 root - INFO - Preview of the DataFrame:
|
| 11 |
+
anime_id genres name ... scored by members image url
|
| 12 |
+
0 32281 Drama, Romance, School, Supernatural Kimi no Na wa. ... 1807089 2597495 https://cdn.myanimelist.net/images/anime/5/870...
|
| 13 |
+
1 5114 Action, Adventure, Drama, Fantasy, Magic, Mili... Fullmetal Alchemist: Brotherhood ... 2020030 3176556 https://cdn.myanimelist.net/images/anime/1208/...
|
| 14 |
+
2 28977 Action, Comedy, Historical, Parody, Samurai, S... Gintama° ... 237957 595767 https://cdn.myanimelist.net/images/anime/3/720...
|
| 15 |
+
3 9253 Sci-Fi, Thriller Steins;Gate ... 1336233 2440369 https://cdn.myanimelist.net/images/anime/1935/...
|
| 16 |
+
4 9969 Action, Comedy, Historical, Parody, Samurai, S... Gintama' ... 226175 525688 https://cdn.myanimelist.net/images/anime/4/503...
|
| 17 |
+
|
| 18 |
+
[5 rows x 18 columns]
|
| 19 |
+
[ 2025-02-03 11:40:50,718 ] 51 root - INFO - Data fetched successfully from Hugging Face.
|
| 20 |
+
[ 2025-02-03 11:40:50,719 ] 40 root - INFO - Fetching data from Hugging Face dataset: krishnaveni76/UserRatings
|
| 21 |
+
[ 2025-02-03 11:42:21,881 ] 48 root - INFO - Shape of the dataframe: (1112830, 4)
|
| 22 |
+
[ 2025-02-03 11:42:21,881 ] 49 root - INFO - Column names: Index(['user_id', 'username', 'anime_id', 'rating'], dtype='object')
|
| 23 |
+
[ 2025-02-03 11:42:21,889 ] 50 root - INFO - Preview of the DataFrame:
|
| 24 |
+
user_id username anime_id rating
|
| 25 |
+
0 357 zhambi 35427 5
|
| 26 |
+
1 357 zhambi 28391 6
|
| 27 |
+
2 357 zhambi 36649 7
|
| 28 |
+
3 357 zhambi 530 6
|
| 29 |
+
4 357 zhambi 37451 7
|
| 30 |
+
[ 2025-02-03 11:42:21,889 ] 51 root - INFO - Data fetched successfully from Hugging Face.
|
| 31 |
+
[ 2025-02-03 11:42:21,894 ] 21 root - INFO - Saving DataFrame to file: Artifacts/02_03_2025_11_40_35/data_ingestion/feature_store/Animes.csv
|
| 32 |
+
[ 2025-02-03 11:42:22,227 ] 25 root - INFO - DataFrame saved successfully to Artifacts/02_03_2025_11_40_35/data_ingestion/feature_store/Animes.csv.
|
| 33 |
+
[ 2025-02-03 11:42:22,227 ] 21 root - INFO - Saving DataFrame to file: Artifacts/02_03_2025_11_40_35/data_ingestion/feature_store/UserRatings.csv
|
| 34 |
+
[ 2025-02-03 11:42:24,690 ] 25 root - INFO - DataFrame saved successfully to Artifacts/02_03_2025_11_40_35/data_ingestion/feature_store/UserRatings.csv.
|
| 35 |
+
[ 2025-02-03 11:42:24,820 ] 46 root - INFO - Data Ingestion completed: DataIngestionArtifact(feature_store_anime_file_path='Artifacts/02_03_2025_11_40_35/data_ingestion/feature_store/Animes.csv', feature_store_userrating_file_path='Artifacts/02_03_2025_11_40_35/data_ingestion/feature_store/UserRatings.csv')
|
| 36 |
+
[ 2025-02-03 11:42:24,820 ] 58 root - INFO - Initiating Data Transformation...
|
| 37 |
+
[ 2025-02-03 11:42:24,821 ] 95 root - INFO - Entering initiate_data_transformation method of DataTransformation class.
|
| 38 |
+
[ 2025-02-03 11:42:29,247 ] 54 root - INFO - Shape of the Merged dataframe:(916416, 21)
|
| 39 |
+
[ 2025-02-03 11:42:29,247 ] 55 root - INFO - Column names: Index(['user_id', 'username', 'anime_id', 'rating', 'genres', 'name',
|
| 40 |
+
'average_rating', 'overview', 'type', 'episodes', 'producers',
|
| 41 |
+
'licensors', 'studios', 'source', 'anime_rating', 'rank', 'popularity',
|
| 42 |
+
'favorites', 'scored by', 'members', 'image url'],
|
| 43 |
+
dtype='object')
|
| 44 |
+
[ 2025-02-03 11:42:31,342 ] 81 root - INFO - Shape of the Merged dataframe:(862804, 8)
|
| 45 |
+
[ 2025-02-03 11:42:31,342 ] 82 root - INFO - Column names: Index(['user_id', 'anime_id', 'rating', 'genres', 'name', 'average_rating',
|
| 46 |
+
'anime_rating', 'image url'],
|
| 47 |
+
dtype='object')
|
| 48 |
+
[ 2025-02-03 11:42:31,361 ] 83 root - INFO - Preview of the merged DataFrame:
|
| 49 |
+
user_id anime_id rating ... average_rating anime_rating image url
|
| 50 |
+
0 357 28391 6 ... 6.65 PG-13 - Teens 13 or older https://cdn.myanimelist.net/images/anime/9/778...
|
| 51 |
+
1 357 530 6 ... 7.73 PG-13 - Teens 13 or older https://cdn.myanimelist.net/images/anime/1440/...
|
| 52 |
+
3 357 17895 7 ... 7.74 PG-13 - Teens 13 or older https://cdn.myanimelist.net/images/anime/12/52...
|
| 53 |
+
4 357 27991 7 ... 7.56 PG-13 - Teens 13 or older https://cdn.myanimelist.net/images/anime/6/761...
|
| 54 |
+
5 357 31798 9 ... 7.38 PG-13 - Teens 13 or older https://cdn.myanimelist.net/images/anime/6/784...
|
| 55 |
+
|
| 56 |
+
[5 rows x 8 columns]
|
| 57 |
+
[ 2025-02-03 11:42:31,399 ] 21 root - INFO - Saving DataFrame to file: Artifacts/02_03_2025_11_40_35/data_transformation/transformed/Anime_UserRatings.csv
|
| 58 |
+
[ 2025-02-03 11:42:38,424 ] 25 root - INFO - DataFrame saved successfully to Artifacts/02_03_2025_11_40_35/data_transformation/transformed/Anime_UserRatings.csv.
|
| 59 |
+
[ 2025-02-03 11:42:38,583 ] 65 root - INFO - Data Transformation completed: DataTransformationArtifact(merged_file_path='Artifacts/02_03_2025_11_40_35/data_transformation/transformed/Anime_UserRatings.csv')
|
| 60 |
+
[ 2025-02-03 11:42:38,584 ] 77 root - INFO - Initiating Collaborative Model Training...
|
| 61 |
+
[ 2025-02-03 11:42:38,584 ] 42 root - INFO - Loading transformed data...
|
| 62 |
+
[ 2025-02-03 11:42:38,585 ] 42 root - INFO - Loading CSV data from file: Artifacts/02_03_2025_11_40_35/data_transformation/transformed/Anime_UserRatings.csv
|
| 63 |
+
[ 2025-02-03 11:42:40,977 ] 44 root - INFO - CSV file loaded successfully.
|
| 64 |
+
[ 2025-02-03 11:42:40,979 ] 26 root - INFO - Initializing CollaborativeAnimeRecommender
|
| 65 |
+
[ 2025-02-03 11:42:44,796 ] 45 root - INFO - Data preparation completed...
|
| 66 |
+
[ 2025-02-03 11:42:44,796 ] 75 root - INFO - Training and saving KNN user-based model...
|
| 67 |
+
[ 2025-02-03 11:42:44,797 ] 78 root - INFO - Training KNN model
|
| 68 |
+
[ 2025-02-03 11:42:44,987 ] 82 root - INFO - KNN model training completed
|
| 69 |
+
[ 2025-02-03 11:42:44,988 ] 59 root - INFO - Entered the save_model method.
|
| 70 |
+
[ 2025-02-03 11:42:45,031 ] 63 root - INFO - Model saved successfully to Artifacts/02_03_2025_11_40_35/trained_models/collaborative_recommenders/userbasedknn.pkl.
|
| 71 |
+
[ 2025-02-03 11:42:45,031 ] 79 root - INFO - Loading pre-trained user-based KNN model...
|
| 72 |
+
[ 2025-02-03 11:42:45,031 ] 79 root - INFO - Attempting to load object from Artifacts/02_03_2025_11_40_35/trained_models/collaborative_recommenders/userbasedknn.pkl
|
| 73 |
+
[ 2025-02-03 11:42:45,031 ] 85 root - INFO - Object loaded successfully.
|
| 74 |
+
[ 2025-02-03 11:42:45,157 ] 253 root - INFO - Shape of filtered df: (10, 8)
|
| 75 |
+
[ 2025-02-03 11:42:45,171 ] 84 root - INFO - User Based recommendations: Anime Name Image URL Genres Rating
|
| 76 |
+
0 Little Busters! Refrain https://cdn.myanimelist.net/images/anime/10/55... Comedy, Drama, Romance, School, Slice of Life,... 8.19
|
| 77 |
+
1 Chuunibyou demo Koi ga Shitai! Ren https://cdn.myanimelist.net/images/anime/7/566... Comedy, Drama, Romance, School, Slice of Life 7.55
|
| 78 |
+
2 Clannad: Mou Hitotsu no Sekai, Tomoyo-hen https://cdn.myanimelist.net/images/anime/12/19... Drama, Romance, School, Slice of Life 7.93
|
| 79 |
+
3 DearS https://cdn.myanimelist.net/images/anime/1207/... Comedy, Ecchi, Harem, Romance, Sci-Fi 6.56
|
| 80 |
+
4 FLCL https://cdn.myanimelist.net/images/anime/7/773... Action, Comedy, Dementia, Mecha, Parody, Sci-Fi 8.03
|
| 81 |
+
5 H2O: Footprints in the Sand https://cdn.myanimelist.net/images/anime/1762/... Harem, Romance, School 7.00
|
| 82 |
+
6 Hatsukoi Limited. https://cdn.myanimelist.net/images/anime/7/155... Comedy, Romance, School, Shounen 7.27
|
| 83 |
+
7 Motto To LOVE-Ru https://cdn.myanimelist.net/images/anime/4/598... Comedy, Ecchi, Harem, School, Sci-Fi, Shounen 7.28
|
| 84 |
+
8 Overlord https://cdn.myanimelist.net/images/anime/7/880... Action, Adventure, Fantasy, Game, Magic, Super... 7.91
|
| 85 |
+
9 No.6 https://cdn.myanimelist.net/images/anime/1474/... Action, Sci-Fi 7.56
|
| 86 |
+
[ 2025-02-03 11:42:45,315 ] 84 root - INFO - Collaborative Model Training completed: CollaborativeModelArtifact(svd_file_path=None, item_based_knn_file_path=None, user_based_knn_file_path='Artifacts/02_03_2025_11_40_35/trained_models/collaborative_recommenders/userbasedknn.pkl')
|
| 87 |
+
[ 2025-02-03 11:42:45,315 ] 96 root - INFO - Initiating Content-Based Model Training...
|
| 88 |
+
[ 2025-02-03 11:42:45,316 ] 37 root - INFO - Loading ingested data...
|
| 89 |
+
[ 2025-02-03 11:42:45,316 ] 42 root - INFO - Loading CSV data from file: Artifacts/02_03_2025_11_40_35/data_ingestion/feature_store/Animes.csv
|
| 90 |
+
[ 2025-02-03 11:42:45,499 ] 44 root - INFO - CSV file loaded successfully.
|
| 91 |
+
[ 2025-02-03 11:42:45,500 ] 39 root - INFO - Training ContentBasedRecommender model...
|
| 92 |
+
[ 2025-02-03 11:43:06,457 ] 35 root - INFO - Saving model to Artifacts/02_03_2025_11_40_35/trained_models/content_based_recommenders/cosine_similarity.pkl
|
| 93 |
+
[ 2025-02-03 11:43:41,225 ] 39 root - INFO - Content recommender Model saved successfully
|
| 94 |
+
[ 2025-02-03 11:43:41,244 ] 46 root - INFO - Model saved successfully.
|
| 95 |
+
[ 2025-02-03 11:43:41,245 ] 48 root - INFO - Loading saved model to get recommendations...
|
| 96 |
+
[ 2025-02-03 11:43:41,250 ] 46 root - INFO - Loading model from Artifacts/02_03_2025_11_40_35/trained_models/content_based_recommenders/cosine_similarity.pkl
|
| 97 |
+
[ 2025-02-03 11:44:23,930 ] 50 root - INFO - Model loaded successfully
|
| 98 |
+
[ 2025-02-03 11:44:24,814 ] 64 root - INFO - Recommendations generated successfully
|
| 99 |
+
[ 2025-02-03 11:44:27,761 ] 50 root - INFO - Cosine similarity recommendations: Anime name Image URL Genres Rating
|
| 100 |
+
0 One Piece: Episode of Merry - Mou Hitori no Na... https://cdn.myanimelist.net/images/anime/9/610... Action, Adventure, Comedy, Drama, Fantasy, Sho... 8.19
|
| 101 |
+
1 One Piece: Episode of Nami - Koukaishi no Nami... https://cdn.myanimelist.net/images/anime/5/414... Action, Adventure, Comedy, Drama, Fantasy, Sho... 8.12
|
| 102 |
+
2 One Piece: Episode of Sabo - 3 Kyoudai no Kizu... https://cdn.myanimelist.net/images/anime/1373/... Action, Adventure, Comedy, Drama, Fantasy, Sho... 7.71
|
| 103 |
+
3 One Piece Film: Strong World https://cdn.myanimelist.net/images/anime/1192/... Action, Adventure, Comedy, Drama, Fantasy, Sho... 8.08
|
| 104 |
+
4 One Piece Film: Z https://cdn.myanimelist.net/images/anime/6/442... Action, Adventure, Comedy, Drama, Fantasy, Sho... 8.14
|
| 105 |
+
5 One Piece Film: Gold https://cdn.myanimelist.net/images/anime/12/81... Action, Adventure, Comedy, Drama, Fantasy, Sho... 7.9
|
| 106 |
+
6 One Piece: Heart of Gold https://cdn.myanimelist.net/images/anime/1673/... Action, Adventure, Comedy, Drama, Fantasy, Sho... 7.5
|
| 107 |
+
7 Digimon Frontier https://cdn.myanimelist.net/images/anime/1048/... Action, Adventure, Comedy, Drama, Fantasy, Sho... 7.15
|
| 108 |
+
8 Digimon Tamers https://cdn.myanimelist.net/images/anime/7/736... Adventure, Comedy, Drama, Fantasy, Shounen 7.63
|
| 109 |
+
9 Digimon Savers https://cdn.myanimelist.net/images/anime/1415/... Adventure, Comedy, Drama, Fantasy, Shounen 6.95
|
| 110 |
+
[ 2025-02-03 11:44:29,361 ] 103 root - INFO - Content-Based Model Training completed: ContentBasedModelArtifact(cosine_similarity_model_file_path='Artifacts/02_03_2025_11_40_35/trained_models/content_based_recommenders/cosine_similarity.pkl')
|
| 111 |
+
[ 2025-02-03 11:44:29,366 ] 113 root - INFO - Initiating Popularity-Based Filtering...
|
| 112 |
+
[ 2025-02-03 11:44:29,371 ] 41 root - INFO - Loading transformed data...
|
| 113 |
+
[ 2025-02-03 11:44:29,376 ] 42 root - INFO - Loading CSV data from file: Artifacts/02_03_2025_11_40_35/data_ingestion/feature_store/Animes.csv
|
| 114 |
+
[ 2025-02-03 11:44:30,775 ] 44 root - INFO - CSV file loaded successfully.
|
| 115 |
+
[ 2025-02-03 11:44:30,776 ] 17 root - INFO - Initializing PopularityBasedFiltering class
|
| 116 |
+
[ 2025-02-03 11:44:30,905 ] 29 root - INFO - Fetching top 10 most popular animes
|
| 117 |
+
[ 2025-02-03 11:44:31,089 ] 48 root - INFO - Popular Anime recommendations: Anime name Image URL Genres Rating
|
| 118 |
+
0 Shingeki no Kyojin https://cdn.myanimelist.net/images/anime/10/47... Action, Drama, Fantasy, Shounen, Super Power 8.54
|
| 119 |
+
1 Death Note https://cdn.myanimelist.net/images/anime/9/945... Mystery, Police, Psychological, Supernatural, ... 8.62
|
| 120 |
+
2 Fullmetal Alchemist: Brotherhood https://cdn.myanimelist.net/images/anime/1208/... Action, Adventure, Drama, Fantasy, Magic, Mili... 9.10
|
| 121 |
+
3 One Punch Man https://cdn.myanimelist.net/images/anime/12/76... Action, Comedy, Parody, Sci-Fi, Seinen, Super ... 8.50
|
| 122 |
+
4 Sword Art Online https://cdn.myanimelist.net/images/anime/11/39... Action, Adventure, Fantasy, Game, Romance 7.20
|
| 123 |
+
5 Boku no Hero Academia https://cdn.myanimelist.net/images/anime/10/78... Action, Comedy, School, Shounen, Super Power 7.89
|
| 124 |
+
6 Naruto https://cdn.myanimelist.net/images/anime/13/17... Action, Comedy, Martial Arts, Shounen, Super P... 7.99
|
| 125 |
+
7 Tokyo Ghoul https://cdn.myanimelist.net/images/anime/1498/... Action, Drama, Horror, Mystery, Psychological,... 7.79
|
| 126 |
+
8 Hunter x Hunter (2011) https://cdn.myanimelist.net/images/anime/1337/... Action, Adventure, Shounen, Super Power 9.04
|
| 127 |
+
9 Kimi no Na wa. https://cdn.myanimelist.net/images/anime/5/870... Drama, Romance, School, Supernatural 8.85
|
| 128 |
+
[ 2025-02-03 11:44:31,094 ] 116 root - INFO - Popularity-Based Filtering completed.
|
| 129 |
+
[ 2025-02-03 11:44:31,094 ] 141 root - INFO - Training Pipeline executed successfully.
|
run_pipeline.py
CHANGED
|
@@ -1,53 +1,68 @@
|
|
| 1 |
import sys
|
|
|
|
| 2 |
from anime_recommender.loggers.logging import logging
|
| 3 |
from anime_recommender.exception.exception import AnimeRecommendorException
|
| 4 |
-
from anime_recommender.components.data_ingestion import DataIngestion
|
| 5 |
-
from anime_recommender.entity.config_entity import TrainingPipelineConfig,DataIngestionConfig,DataTransformationConfig,CollaborativeModelConfig,ContentBasedModelConfig
|
| 6 |
-
from anime_recommender.components.data_transformation import DataTransformation
|
| 7 |
-
from anime_recommender.components.collaborative_recommender import CollaborativeModelTrainer
|
| 8 |
-
from anime_recommender.components.content_based_recommender import ContentBasedModelTrainer
|
| 9 |
-
from anime_recommender.components.top_anime_recommenders import PopularityBasedRecommendor
|
| 10 |
-
|
| 11 |
|
| 12 |
if __name__ == "__main__":
|
| 13 |
try:
|
| 14 |
-
|
| 15 |
-
|
| 16 |
-
|
| 17 |
-
logging.info("Initiating Data Ingestion.")
|
| 18 |
-
data_ingestion_artifact = data_ingestion.ingest_data()
|
| 19 |
-
logging.info(f"Data ingestion completed.")
|
| 20 |
-
print(data_ingestion_artifact)
|
| 21 |
-
|
| 22 |
-
# Data Transformation
|
| 23 |
-
data_transformation_config = DataTransformationConfig(training_pipeline_config)
|
| 24 |
-
data_transformation = DataTransformation(data_ingestion_artifact,data_transformation_config)
|
| 25 |
-
logging.info("Initiating Data Transformation.")
|
| 26 |
-
data_transformation_artifact = data_transformation.initiate_data_transformation()
|
| 27 |
-
logging.info("Data Transformation Completed.")
|
| 28 |
-
print(data_transformation_artifact)
|
| 29 |
-
|
| 30 |
-
# Collaborative Model Training
|
| 31 |
-
collaborative_model_trainer_config = CollaborativeModelConfig(training_pipeline_config)
|
| 32 |
-
collaborative_model_trainer = CollaborativeModelTrainer(collaborative_model_trainer_config= collaborative_model_trainer_config,data_transformation_artifact=data_transformation_artifact)
|
| 33 |
-
logging.info("Initiating Collaborative Model training.")
|
| 34 |
-
collaborative_model_trainer_artifact = collaborative_model_trainer.initiate_model_trainer(model_type='user_knn')
|
| 35 |
-
logging.info("Collaborative Model training completed.")
|
| 36 |
-
print(collaborative_model_trainer_artifact)
|
| 37 |
-
|
| 38 |
-
# Content Based Model Training
|
| 39 |
-
content_based_model_trainer_config = ContentBasedModelConfig(training_pipeline_config)
|
| 40 |
-
content_based_model_trainer = ContentBasedModelTrainer(content_based_model_trainer_config=content_based_model_trainer_config,data_ingestion_artifact=data_ingestion_artifact)
|
| 41 |
-
logging.info("Initiating Content Based Model training.")
|
| 42 |
-
content_based_model_trainer_artifact = content_based_model_trainer.initiate_model_trainer()
|
| 43 |
-
logging.info("Content Based Model training completed.")
|
| 44 |
-
print(content_based_model_trainer_artifact)
|
| 45 |
-
|
| 46 |
-
# Popularity Based Filtering
|
| 47 |
-
logging.info("Initiating Popularity based filtering.")
|
| 48 |
-
filtering = PopularityBasedRecommendor(data_ingestion_artifact=data_ingestion_artifact)
|
| 49 |
-
popularity_recommendations = filtering.initiate_model_trainer(filter_type='top_avg_rated')
|
| 50 |
-
logging.info("Popularity based filtering completed.")
|
| 51 |
-
|
| 52 |
except Exception as e:
|
| 53 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
import sys
|
| 2 |
+
from anime_recommender.pipelines.training_pipeline import TrainingPipeline
|
| 3 |
from anime_recommender.loggers.logging import logging
|
| 4 |
from anime_recommender.exception.exception import AnimeRecommendorException
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 5 |
|
| 6 |
if __name__ == "__main__":
|
| 7 |
try:
|
| 8 |
+
logging.info("Starting the Anime Recommendation System Training Pipeline...")
|
| 9 |
+
pipeline = TrainingPipeline()
|
| 10 |
+
pipeline.run_pipeline()
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 11 |
except Exception as e:
|
| 12 |
+
logging.error(f"Pipeline execution failed: {str(e)}")
|
| 13 |
+
raise AnimeRecommendorException(e, sys)
|
| 14 |
+
|
| 15 |
+
|
| 16 |
+
# import sys
|
| 17 |
+
# from anime_recommender.loggers.logging import logging
|
| 18 |
+
# from anime_recommender.exception.exception import AnimeRecommendorException
|
| 19 |
+
# from anime_recommender.components.data_ingestion import DataIngestion
|
| 20 |
+
# from anime_recommender.entity.config_entity import TrainingPipelineConfig,DataIngestionConfig,DataTransformationConfig,CollaborativeModelConfig,ContentBasedModelConfig
|
| 21 |
+
# from anime_recommender.components.data_transformation import DataTransformation
|
| 22 |
+
# from anime_recommender.components.collaborative_recommender import CollaborativeModelTrainer
|
| 23 |
+
# from anime_recommender.components.content_based_recommender import ContentBasedModelTrainer
|
| 24 |
+
# from anime_recommender.components.top_anime_recommenders import PopularityBasedRecommendor
|
| 25 |
+
|
| 26 |
+
|
| 27 |
+
# if __name__ == "__main__":
|
| 28 |
+
# try:
|
| 29 |
+
# training_pipeline_config = TrainingPipelineConfig()
|
| 30 |
+
# data_ingestion_config = DataIngestionConfig(training_pipeline_config)
|
| 31 |
+
# data_ingestion = DataIngestion(data_ingestion_config)
|
| 32 |
+
# logging.info("Initiating Data Ingestion.")
|
| 33 |
+
# data_ingestion_artifact = data_ingestion.ingest_data()
|
| 34 |
+
# logging.info(f"Data ingestion completed.")
|
| 35 |
+
# print(data_ingestion_artifact)
|
| 36 |
+
|
| 37 |
+
# # Data Transformation
|
| 38 |
+
# data_transformation_config = DataTransformationConfig(training_pipeline_config)
|
| 39 |
+
# data_transformation = DataTransformation(data_ingestion_artifact,data_transformation_config)
|
| 40 |
+
# logging.info("Initiating Data Transformation.")
|
| 41 |
+
# data_transformation_artifact = data_transformation.initiate_data_transformation()
|
| 42 |
+
# logging.info("Data Transformation Completed.")
|
| 43 |
+
# print(data_transformation_artifact)
|
| 44 |
+
|
| 45 |
+
# # Collaborative Model Training
|
| 46 |
+
# collaborative_model_trainer_config = CollaborativeModelConfig(training_pipeline_config)
|
| 47 |
+
# collaborative_model_trainer = CollaborativeModelTrainer(collaborative_model_trainer_config= collaborative_model_trainer_config,data_transformation_artifact=data_transformation_artifact)
|
| 48 |
+
# logging.info("Initiating Collaborative Model training.")
|
| 49 |
+
# collaborative_model_trainer_artifact = collaborative_model_trainer.initiate_model_trainer(model_type='user_knn')
|
| 50 |
+
# logging.info("Collaborative Model training completed.")
|
| 51 |
+
# print(collaborative_model_trainer_artifact)
|
| 52 |
+
|
| 53 |
+
# # Content Based Model Training
|
| 54 |
+
# content_based_model_trainer_config = ContentBasedModelConfig(training_pipeline_config)
|
| 55 |
+
# content_based_model_trainer = ContentBasedModelTrainer(content_based_model_trainer_config=content_based_model_trainer_config,data_ingestion_artifact=data_ingestion_artifact)
|
| 56 |
+
# logging.info("Initiating Content Based Model training.")
|
| 57 |
+
# content_based_model_trainer_artifact = content_based_model_trainer.initiate_model_trainer()
|
| 58 |
+
# logging.info("Content Based Model training completed.")
|
| 59 |
+
# print(content_based_model_trainer_artifact)
|
| 60 |
+
|
| 61 |
+
# # Popularity Based Filtering
|
| 62 |
+
# logging.info("Initiating Popularity based filtering.")
|
| 63 |
+
# filtering = PopularityBasedRecommendor(data_ingestion_artifact=data_ingestion_artifact)
|
| 64 |
+
# popularity_recommendations = filtering.initiate_model_trainer(filter_type='top_avg_rated')
|
| 65 |
+
# logging.info("Popularity based filtering completed.")
|
| 66 |
+
|
| 67 |
+
# except Exception as e:
|
| 68 |
+
# raise AnimeRecommendorException(e, sys)
|