Update README.md

README.md

@@ -16,7 +16,9 @@ TRL supports the DPO Trainer for training language models from preference data,
  target_modules=[ "gate_proj", "up_proj", "down_proj"]
 
 ```
-sample codeimport torch
+sample code
+```
+import torch
 from transformers import AutoTokenizer, AutoModelForCausalLM
 import math
 
@@ -40,4 +42,3 @@ while len(prompt) > 0:
   
 ```
 
-```