Update README.md

README.md

@@ -20,19 +20,96 @@ It achieves the following results on the evaluation set:
 - Loss: 3.0259
 - Accuracy: 0.4040
 
-## Model description
+Model Description
+This model is a fine-tuned version of ai-forever/rugpt3small_based_on_gpt2, designed for causal language modeling tasks. It has been trained on a custom dataset to generate coherent and contextually relevant text.
 
-More information needed
+Training Details
+Training Epochs: 29.86
 
-## Intended uses & limitations
+Total FLOPs: 8,153,103 GF
 
-More information needed
+Training Loss: 3.8147
 
-## Training and evaluation data
+Training Runtime: 35 minutes and 43.75 seconds
 
-More information needed
+Number of Training Samples: 291
+
+Training Samples per Second: 4.072
+
+Training Steps per Second: 0.056
+
+Evaluation Metrics
+Evaluation Epoch: 29.86
+
+Evaluation Accuracy: 40.4%
+
+Evaluation Loss: 3.0259
+
+Evaluation Runtime: 0.12 seconds
+
+Number of Evaluation Samples: 1
+
+Evaluation Samples per Second: 8.08
+
+Evaluation Steps per Second: 8.08
+
+Perplexity: 20.6125
+
+Intended Use
+This model is intended for text generation tasks where coherent and contextually appropriate responses are required. It can be used in applications such as chatbots, content creation, and more.
+
+Limitations
+The model has been trained on a limited dataset (291 samples), which may affect its generalization capabilities.
+
+The evaluation accuracy of approximately 40% indicates that the model may not perform optimally across all contexts.
+
+The perplexity score suggests room for improvement in generating more confident predictions.
+
+Future Work
+To enhance the performance of this model, consider the following:
+
+Increase the size and diversity of the training dataset.
+
+Experiment with additional training epochs or different hyperparameters.
+
+Evaluate the model on a broader set of examples to better assess its capabilities.
 
 ## Training procedure
+## [Training Procedure](pplx://action/followup)
+
+The model was trained using the `transformers` library and the `run_clm.py` script. Here's a summary of the training process:
+
+*   **[Model](pplx://action/followup):** `ai-forever/rugpt3small_based_on_gpt2` (a Russian language GPT-2 model).
+*   **[Objective](pplx://action/followup):** Causal Language Modeling (text generation).
+*   **[Hardware](pplx://action/followup):** Google Colab with a single CUDA-enabled GPU.
+*   **[Mixed Precision](pplx://action/followup):** FP16 training was enabled to reduce memory footprint and potentially improve training speed.
+*   **[Optimizer](pplx://action/followup):** AdamW (`adamw_torch`) was used as the optimizer.
+*   **[Learning Rate](pplx://action/followup):** The learning rate was set to `3e-5`.
+*   **[Warmup](pplx://action/followup):** A linear warmup schedule with `500` warmup steps was used.
+*   **[Training Data](pplx://action/followup):** Custom text dataset loaded from `
+  The model was trained on a custom text dataset loaded from the following sources using the `plain_text` dataset configuration:
+
+*   **Training set:**  Aristotle's major works. (32,835 examples)
+    *   Аристотель. Категории
+    *   Аристотель. Никомахова этика
+    *   Аристотель. Физика
+    *   Аристотель. Метафизика
+    *   Аристотель. Риторика
+    *   Аристотель. Поэтика
+    *   **[Validation Data](pplx://action/followup):** Custom text dataset loaded from `- Аристотель. Никомахова этика ttps://lib.ru/POEEAST/ARISTOTEL/nikomah.txt` using the `plain_text` dataset configuration. The validation set contained 111 examples.
+
+*   **Validation set:** Aristotle. Никомахова этика (111 examples)
+
+  
+*   
+*   **[Batch Size](pplx://action/followup):** A per-device batch size of `8` was used with a gradient accumulation size of `8`, resulting in an effective batch size of 64.
+*   **[Sequence Length](pplx://action/followup):** The maximum sequence length (block size) was set to `2048`.
+*   **[Gradient Checkpointing](pplx://action/followup):** Enabled to reduce memory consumption.
+*   **[Epochs](pplx://action/followup):** Trained for `30` epochs. The training data was passed over 30 times.
+*   **[Evaluation](pplx://action/followup):** Evaluation was performed every `1000` steps using the validation dataset.
+*   **[Logging](pplx://action/followup):** Training progress and metrics were logged every `100` steps to TensorBoard and Weights & Biases (WandB).
+*   **[Checkpoints](pplx://action/followup):** Model checkpoints were saved every `1000` steps, with a limit of `3` saved checkpoints.
+
 
 ### Training hyperparameters