kaikaidai commited on
Commit
2642da4
·
verified ·
1 Parent(s): b353f07

Update prompts.py

Browse files
Files changed (1) hide show
  1. prompts.py +4 -4
prompts.py CHANGED
@@ -94,14 +94,14 @@ Score 5: {score5_desc}
94
  # Judge system prompt for non-Prometheus models
95
  JUDGE_SYSTEM_PROMPT = """Please act as an impartial judge and evaluate based on the user's instruction. Your output format should strictly adhere to JSON as follows: {"feedback": "<write feedback>", "result": <numerical score>}. Ensure the output is valid JSON, without additional formatting or explanations."""
96
 
97
- ATLA_PROMPT = """You are tasked with evaluating a response based on a given instruction (which may contain an Input) and a scoring rubric that serve as the evaluation standard. Provide a comprehensive feedback on the response quality strictly adhering to the scoring rubric, without any general evaluation. Follow this with a score between 1 and 5, referring to the scoring rubric. Avoid generating any additional opening, closing, or explanations.
98
  Here are some rules of the evaluation:
99
  (1) You should prioritize evaluating whether the response satisfies the provided rubric. The basis of your score should depend exactly on the rubric. However, the response does not need to explicitly address points raised in the rubric. Rather, evaluate the response based on the criteria outlined in the rubric.
100
 
101
  Your reply should strictly follow this format:
102
  **Reasoning:** <Your feedback>
103
 
104
- **Result:** <an integer between 1 and 5>
105
 
106
  Here is the data:
107
 
@@ -118,7 +118,7 @@ ATLA_PROMPT = """You are tasked with evaluating a response based on a given inst
118
  Score Rubrics:
119
  {eval_criteria}"""
120
 
121
- ATLA_PROMPT_WITH_REFERENCE = """You are tasked with evaluating a response based on a given instruction (which may contain an Input) and a scoring rubric and reference answer that serve as the evaluation standard. Provide a comprehensive feedback on the response quality strictly adhering to the scoring rubric, without any general evaluation. Follow this with a score between 1 and 5, referring to the scoring rubric. Avoid generating any additional opening, closing, or explanations.
122
 
123
  Here are some rules of the evaluation:
124
  (1) You should prioritize evaluating whether the response satisfies the provided rubric. The basis of your score should depend exactly on the rubric. However, the response does not need to explicitly address points raised in the rubric. Rather, evaluate the response based on the criteria outlined in the rubric.
@@ -126,7 +126,7 @@ ATLA_PROMPT_WITH_REFERENCE = """You are tasked with evaluating a response based
126
  Your reply should strictly follow this format:
127
  **Reasoning:** <Your feedback>
128
 
129
- **Result:** <an integer between 1 and 5>
130
 
131
  Here is the data:
132
 
 
94
  # Judge system prompt for non-Prometheus models
95
  JUDGE_SYSTEM_PROMPT = """Please act as an impartial judge and evaluate based on the user's instruction. Your output format should strictly adhere to JSON as follows: {"feedback": "<write feedback>", "result": <numerical score>}. Ensure the output is valid JSON, without additional formatting or explanations."""
96
 
97
+ ATLA_PROMPT = """You are tasked with evaluating a response based on a given instruction (which may contain an Input) and a scoring rubric that serve as the evaluation standard. Provide a comprehensive feedback on the response quality strictly adhering to the scoring rubric, without any general evaluation. Follow this with a score integer, referring to the scoring rubric. Avoid generating any additional opening, closing, or explanations.
98
  Here are some rules of the evaluation:
99
  (1) You should prioritize evaluating whether the response satisfies the provided rubric. The basis of your score should depend exactly on the rubric. However, the response does not need to explicitly address points raised in the rubric. Rather, evaluate the response based on the criteria outlined in the rubric.
100
 
101
  Your reply should strictly follow this format:
102
  **Reasoning:** <Your feedback>
103
 
104
+ **Result:** <Your score>
105
 
106
  Here is the data:
107
 
 
118
  Score Rubrics:
119
  {eval_criteria}"""
120
 
121
+ ATLA_PROMPT_WITH_REFERENCE = """You are tasked with evaluating a response based on a given instruction (which may contain an Input) and a scoring rubric and reference answer that serve as the evaluation standard. Provide a comprehensive feedback on the response quality strictly adhering to the scoring rubric, without any general evaluation. Follow this with a score, referring to the scoring rubric. Avoid generating any additional opening, closing, or explanations.
122
 
123
  Here are some rules of the evaluation:
124
  (1) You should prioritize evaluating whether the response satisfies the provided rubric. The basis of your score should depend exactly on the rubric. However, the response does not need to explicitly address points raised in the rubric. Rather, evaluate the response based on the criteria outlined in the rubric.
 
126
  Your reply should strictly follow this format:
127
  **Reasoning:** <Your feedback>
128
 
129
+ **Result:** <Your score>
130
 
131
  Here is the data:
132