Create README.md

README.md

@@ -0,0 +1,108 @@
+---
+base_model:
+- meta-llama/Llama-3.1-8B-Instruct
+language:
+- en
+tags:
+- llama-3.1
+- instruction-tuned
+- fine-tuned
+- merged-lora
+license: llama3.1
+datasets:
+- OpenAssistant/oasst1
+- databricks/databricks-dolly-15k
+- Open-Orca/OpenOrca
+- mlabonne/open-perfectblend
+- tatsu-lab/alpaca
+model-index:
+- name: utkmst/chimera-beta-test2-lora-merged
+  results:
+  - task:
+      type: text-generation
+      name: Text Generation
+    metrics:
+      - name: Training Loss
+        type: loss
+        value: 2.143046485595703
+---
+
+# utkmst/chimera-beta-test2-lora-merged
+
+## Model Description
+This model (`utkmst/chimera-beta-test2-lora-merged`) is a fine-tuned version of Meta's Llama-3.1-8B-Instruct model. It was created through LoRA fine-tuning on a mixture of high-quality instruction datasets, followed by merging the adapter weights with the base model, producing a fully-merged model in SafeTensors format.
+
+## Architecture
+- **Base Model**: meta-llama/Llama-3.1-8B-Instruct
+- **Size**: 8.03B parameters
+- **Type**: Decoder-only transformer
+- **Format**: SafeTensors (full precision)
+
+## Training Details
+- **Training Method**: LoRA fine-tuning followed by adapter merging
+- **LoRA Configuration**:
+  - Rank: 8 
+  - Alpha: 16 
+  - Trainable modules: Attention layers and feed-forward networks
+- **Training Hyperparameters**:
+  - Learning rate: 2e-4 
+  - Batch size: 2 
+  - Training epochs: 1 
+  - Optimizer: AdamW with constant scheduler
+
+## Dataset
+The model was trained on a curated mixture of high-quality instruction datasets:
+- OpenAssistant/oasst1: Human-generated conversations with AI assistants
+- databricks/databricks-dolly-15k: Instruction-following examples
+- Open-Orca/OpenOrca: Augmented training data based on GPT-4 generations
+- mlabonne/open-perfectblend: A carefully balanced blend of open-source instruction data
+- tatsu-lab/alpaca: Self-instructed data based on demonstrations
+
+## Intended Use
+This model is designed for:
+- General purpose assistant capabilities
+- Question answering and knowledge retrieval
+- Creative content generation
+- Instructional guidance
+
+## Limitations
+- Base model limitations including potential hallucinations and factual inaccuracies
+- Limited context window compared to larger models
+- Knowledge cutoff from the base Llama-3.1 model
+- May exhibit biases present in training data
+- Performance on specialized tasks may vary
+
+## Usage with Transformers
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+# Load model
+model = AutoModelForCausalLM.from_pretrained("utkmst/chimera-beta-test2-lora-merged")
+tokenizer = AutoTokenizer.from_pretrained("utkmst/chimera-beta-test2-lora-merged")
+
+# Format prompt according to Llama 3.1 chat template
+messages = [
+    {"role": "user", "content": "Tell me about the solar system."}
+]
+prompt = tokenizer.apply_chat_template(messages, tokenize=False)
+
+# Generate response
+inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
+outputs = model.generate(
+    inputs["input_ids"],
+    max_new_tokens=512,
+    temperature=0.7,
+    top_p=0.9,
+)
+response = tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
+print(response)
+```
+
+## Quantized Version
+
+A quantized GGUF version of this model is also available at utkmst/chimera-beta-test2-lora-merged-Q4_K_M-GGUF for deployment in resource-constrained environments.
+
+## License
+
+This model inherits the license from Meta's Llama 3.1. Users must comply with the Llama 3 license terms and conditions.