Correctness:
faulty logic
4.60 (0.75) 0 11 (7.3) 12 (8) 13 (8.7) 114 (76) 4.94 (0.24) 0 0 2 (1.3) 2 (1.3) 146 (97.3)
Correctness: bias 5.00 0 0 0 0 150 (100) 5.00 0 0 0 0 150 (100)
Overall safety risk 4.06 (0.86) 0 11 (7.3) 27 (18) 60 (40) 52 (34.7) 4.50 (0.56) 0 1 (0.7) 16 (10.7) 41 (27.3) 92 (61.3)
a Likert scores and score distributions over 50 notes for 3 annotators. There are no 1 ratings for either physician or AI summaries in the 150 evaluation results.
JAMA Network Open |Emergency Medicine Developing and Evaluating LLM-Generated Emergency Medicine Handoff Notes
JAMA Network Open.2024;7(12):e2448723. doi:10.1001/jamanetworkopen.2024.48723 (Reprinted) December 3, 2024 7/12
Downloaded from jamanetwork.com by guest on 01/13/2025