Upload 6 files

README.md

@@ -1,8 +1,120 @@
 ---
 language:
-- ko
+  - ko
 tags:
-- cnn
-- audio
-- mfcc
----
+  - cnn
+  - audio
+  - mfcc
+  - homecam 
+  - senior
+---
+
+# Model Card for `SilverAudio`
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+## Model Details
+
+- The audio model is a critical component of the `SilverAssistant` system, designed to detect potentially dangerous situations such as falls, cries for help, or signs of violence through audio analysis. Its primary purpose is to enhance user safety while respecting privacy. 
+- By leveraging Mel-Frequency Cepstral Coefficients (MFCC) for feature extraction and Convolutional Neural Networks (CNN) for classification, the model ensures accurate detection without requiring constant video monitoring, preserving user privacy.
+![Audio Model](./pics/audio-overview.png)
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+- **Activity with:** NIPA-Google(2024.10.23-20224.11.08), Kosa Hackathon(2024.12.9)
+- **Model type:** Audio CNN Model
+- **API used:** Keras
+- **Dataset:** [HuggingFace SilverDataset](https://huggingface.co/datasets/SilverAvocado/SilverDataset)
+- **Code:** [GitHub Silver Model Code](https://github.com/silverAvocado/silver-model-code)
+- **Language(s) (NLP):** Korean
+
+## Training Details
+### Dataset Preperation
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+- **HuggingFace:** [HuggingFace SilverDataset](https://huggingface.co/datasets/SilverAvocado/SilverDataset)
+- **Description:**
+    - The dataset used for this audio model consists of `.npy` files containing MFCC features extracted from raw audio recordings. These recordings include various real-world scenarios, such as:
+	    - Criminal activities
+        - Violence
+	    - Falls
+	    - Cries for help
+	    - Normal indoor sounds
+
+    - Feature Extraction Process
+        1.	Audio Collection:
+            - Audio samples were sourced from datasets, such as AI Hub, to ensure coverage of diverse scenarios. 
+            - [AI Hub 위급상황 음성/음향](https://www.aihub.or.kr/aihubdata/data/view.do?currMenu=&topMenu=&aihubDataSe=data&dataSetSn=170)
+            - These include emergency and non-emergency sounds to train the model for accurate classification.
+	    2.	MFCC Extraction:
+	        - The raw audio signals were processed to extract Mel-Frequency Cepstral Coefficients (MFCC).
+	        - The MFCC features effectively capture the frequency characteristics of the audio, making them suitable for sound classification tasks.
+            ![MFCC Output](./pics/mfcc-output.png)
+        3.	Output Format:
+            - The extracted MFCC features are saved as `13 x n` numpy arrays, where:
+            - 13: Represents the number of MFCC coefficients (features).
+            - n: Corresponds to the number of frames in the audio segment.
+        4.	Saved Dataset:
+            - The processed `13 x n` MFCC arrays are stored as `.npy` files, which serve as the direct input to the model.
+
+### Model Description
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+- **Model Structure:**
+    ![Audio Model Structure](./pics/audio-model-structure.png)
+    - Input Layer
+	    - Shape:  (128, 128, 3) 
+	    - Represents the input MFCC features reshaped into a 128x128 image-like format with 3 channels (RGB)
+	    - This transformation helps the CNN process MFCC features as spatial data.
+    - Convolutional Layers (Conv2D)
+	    1.	First Conv2D Layer:
+	        - Extracts low-level features from the MFCC input (e.g., frequency characteristics).
+	        - Applies filters (kernels) to create feature maps by learning patterns in the audio data.
+	    2.	Second Conv2D Layer:
+	        - Further processes the feature maps, learning more complex and hierarchical audio features.
+    - MaxPooling2D Layers
+	    - Located after each Conv2D layer.
+	    - Reduces the spatial dimensions of feature maps, lowering computational complexity and extracting dominant features.
+	    - Helps in preventing overfitting by reducing noise and irrelevant details.
+    - Flatten Layer
+	    - Converts the 2D feature maps into a 1D vector.
+	    - Prepares the data for the dense (fully connected) layers, which act as the classifier.
+    - Fully Connected Layers (Dense)
+	    1.	First Dense Layer:
+	        - Contains 128 neurons.
+	        - Processes the flattened features to learn complex relationships in the data.
+	    2.	Dropout Layer:
+	        - Randomly sets a fraction of input neurons to zero during training.
+	        - Helps in regularization, preventing overfitting by ensuring the model doesn’t rely on specific neurons.
+	    3.	Output Dense Layer:
+	        - Neurons: 2 (for binary classification)
+                - **Class 0: Normal**
+                - **Class 1: Danger**
+	        - Activation: Softmax (to output probabilities for each class).
+- **Model Performance:**
+    ![Model Performance](./pics/audio-performance.png)
+    1. Accuracy and Preprocessing (Table Summary)
+	    - The CNN model achieves the highest accuracy of 98.37% in the 8th configuration.
+	    - Key factors contributing to this performance:
+	        - Input Size: 128 \times 128 \times 3, leveraging image-like MFCC features.
+	        - Padding/Splitting: Audio segments are preprocessed into 5-second splits, ensuring uniform input.
+	        - No Noise Addition: Training without additional noise leads to better feature retention.
+	        - Library: Keras is used for training and architecture implementation.
+	        - Sample Rate: A consistent sample rate of 16,000 Hz was maintained for all preprocessing steps.
+    ![Confusion Matrix](./pics/confusion-matrix.png)
+    2. Confusion Matrix Analysis
+        - High Precision: Minimal false positives suggest the model is very specific when identifying emergencies.
+	    - High Recall: Minimal false negatives indicate that most emergencies are correctly identified.
+
+## Model Usage
+- `Silver Assistant` Project
+    - The audio model is a critical component of the SilverAssistant system, designed to detect potentially dangerous situations such as falls, cries for help, or signs of violence through audio analysis. Its primary purpose is to enhance user safety while respecting privacy
+    - [GitHub SilverAvocado](https://github.com/silverAvocado)
+
+## Conclusion
+- The `SilverAudio` model demonstrates exceptional performance in detecting emergency audio scenarios with high accuracy and reliability, achieving a peak accuracy of 98.37% in the 8th configuration. By leveraging MFCC features and a robust CNN-based architecture, the model effectively classifies audio inputs into predefined categories (normal vs danger). Its ability to operate on preprocessed `.npy` files ensures efficient inference, making it suitable for real-time applications.
+- This model is a vital component of the `SilverAssistant` project, empowering the system to accurately detect critical situations and enable timely interventions. Its real-world applicability, achieved through comprehensive training on diverse datasets such as AI Hub, makes it a powerful tool for enhancing the safety and well-being of vulnerable individuals, particularly the elderly.