sehyun66/Tiny-lama-1.3B-chat-ppo

RLHF with ppo_Trainer and Lora

Hyperparameter

#ppo learning_rate=5e-6, batch_size=32, mini_batch_size=1, horizon=10000, cliprange =0.2, cliprange_value=0.2, lam=0.95, target_kl=2, use_score_scaling = True, log_with='wandb'

#lora r=16, lora_alpha=32, lora_dropout=0.05, bias="none", task_type="CAUSAL_LM",