---
library_name: peft
license: apache-2.0
datasets:
- Abirate/english_quotes
language:
- en
pipeline_tag: text-generation
---
## Base model
bigscience/bloomz-560m

## Training procedure

According to edX Databricks llm102 course

### PromptTuningConfig
 - task_type=TaskType.CAUSAL_LM,
 - prompt_tuning_init=PromptTuningInit.RANDOM,
 - num_virtual_tokens=4,
### TrainingArguments
 - learning_rate= 3e-2, # Higher learning rate than full fine-tuning
 - num_train_epochs=5 # Number of passes to go through the entire fine-tuning dataset 
### Framework versions


- PEFT 0.4.0

### Training output

TrainOutput(global_step=35, training_loss=3.386413792201451, metrics={'train_runtime': 617.1546, 'train_samples_per_second': 0.405, 'train_steps_per_second': 0.057, 'total_flos': 58327152033792.0, 'train_loss': 3.386413792201451, 'epoch': 5.0})