luigi12345 commited on
Commit
d6e21f9
·
verified ·
1 Parent(s): b9ebc0b

Update app.py

Browse files
Files changed (1) hide show
  1. app.py +62 -3
app.py CHANGED
@@ -63,7 +63,7 @@ def multimodal_understanding(image, question, seed, top_p, temperature):
63
  pad_token_id=tokenizer.eos_token_id,
64
  bos_token_id=tokenizer.bos_token_id,
65
  eos_token_id=tokenizer.eos_token_id,
66
- max_new_tokens=512,
67
  do_sample=False if temperature == 0 else True,
68
  use_cache=True,
69
  temperature=temperature,
@@ -193,8 +193,67 @@ with gr.Blocks() as demo:
193
  "doge.png",
194
  ],
195
  [
196
- "Convert the formula into latex code.",
197
- "equation.png",
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
198
  ],
199
  ],
200
  inputs=[question_input, image_input],
 
63
  pad_token_id=tokenizer.eos_token_id,
64
  bos_token_id=tokenizer.bos_token_id,
65
  eos_token_id=tokenizer.eos_token_id,
66
+ max_new_tokens=4000,
67
  do_sample=False if temperature == 0 else True,
68
  use_cache=True,
69
  temperature=temperature,
 
193
  "doge.png",
194
  ],
195
  [
196
+ """Analyze the provided fundus image in exhaustive detail, following the standard ophthalmological protocol for fundus examination. Output an HTML report structured as a formal medical document. The report MUST:
197
+
198
+ 1. **Image Quality Assessment:** Begin with a concise assessment of image quality, noting focus, illumination, field of view, and any artifacts (and their impact on assessability).
199
+
200
+ 2. **Detailed Clinical Findings:** Describe each of the following areas with the utmost precision and specificity, using proper ophthalmological terminology:
201
+ * **Optic Disc:**
202
+ * Size and shape (including any abnormalities).
203
+ * Color (specifically noting any pallor and its location).
204
+ * Cup-to-Disc Ratio (CDR), providing both vertical and horizontal estimates.
205
+ * Neuroretinal Rim: Assess rim thickness in all quadrants (superior, inferior, nasal, temporal). Explicitly state whether the ISNT rule is followed or violated. Describe any notching or focal thinning.
206
+ * Peripapillary Region: Describe the presence/absence of peripapillary atrophy (PPA), differentiating between alpha and beta zones. Note any hemorrhages.
207
+ * **Retinal Vasculature:**
208
+ * Arterioles: Describe caliber (narrowing, dilation), tortuosity, and any focal abnormalities.
209
+ * Venules: Describe caliber, tortuosity, and any abnormalities.
210
+ * Arteriovenous (A/V) Ratio: Estimate the A/V ratio.
211
+ * Crossing Changes: Note any arteriovenous nicking or other crossing abnormalities.
212
+ * Vessel Course: Describe the course of the major vessels, and check for abnormalities.
213
+ * **Macula:**
214
+ * Foveal Reflex: Describe the presence/absence and quality of the foveal reflex.
215
+ * Pigment Changes: Note any pigmentary abnormalities, drusen, or other lesions.
216
+ * Edema/Exudates: Describe any signs of macular edema or exudates.
217
+ * **Peripheral Retina:**
218
+ * Mid-Periphery: Describe any abnormalities (hemorrhages, exudates, tears, etc.).
219
+ * Far Periphery: Note the extent of visualization and any findings.
220
+
221
+ 3. **Differential Diagnosis:** Based solely on the image findings, provide a prioritized differential diagnosis. Include the most likely diagnosis and any other plausible possibilities. For each diagnosis, explain the reasoning based on the observed features.
222
+
223
+ 4. **Diagnostic Confidence:** Indicate the confidence level for the primary diagnosis. List the key image findings that support the diagnosis.
224
+
225
+ 5. **Simulated AI Attention Metrics:** Create a table representing a *simulated* AI attention distribution. This should reflect the expected focus areas for the most likely diagnosis, based on the known importance of different features. Provide percentages for:
226
+ * Optic Disc (Total)
227
+ * Cup
228
+ * Neuroretinal Rim (subdivided by region if significant differences exist)
229
+ * Peripapillary Atrophy
230
+ * Vessels
231
+ * Macula
232
+ * Periphery
233
+
234
+ 6. **Summary and Impression:** Provide a concise summary of the key findings and the overall impression.
235
+
236
+ 7. **Recommendations:**
237
+ * Provide specific, actionable recommendations based on the image findings.
238
+ * If referral is warranted, clearly state the urgency and the type of specialist.
239
+ * List any recommended investigations (e.g., OCT, visual fields).
240
+
241
+ 8. **Disclaimer:** Include a disclaimer stating that the report is based on image analysis alone and does not replace a full clinical examination.
242
+
243
+ 9. **HTML Structure:** Use semantic HTML elements (h1-h3, p, ol, ul, table, div) to create a well-structured, readable report. Include:
244
+ * A report header with a title ("EyeUnit.ai | AI for Ophthalmology") and a logo placeholder.
245
+ * An image comparison section displaying the original fundus image and a placeholder for a heatmap (a canvas element with id "heatmapCanvas"). No actual heatmap generation is required; the canvas is a placeholder.
246
+ * A placeholder for patient information(PATIENT ID, NAME, AGE, DATE OF EXAM)
247
+ * Clearly labeled sections for each part of the analysis.
248
+ * Tables for the "Overall Analysis Coverage" and "AI-Driven Attention Metrics."
249
+
250
+ 10. **CSS Styling:** Apply CSS styles to make the report visually appealing and professional. The report should be suitable for both screen viewing and printing (use a `@media print` block to optimize for print).
251
+ * **Crucial Details:**
252
+ * **PATIENT ID, NAME, AGE and DATE OF EXAM**
253
+
254
+ 11. **Crucial Details:** Output ONLY the complete HTML code. Do not provide any surrounding text or explanations. Focus solely on generating the HTML report.
255
+ 12. **IMG SOURCE:** Use this image as the image source: `<img src="https://i.imgur.com/kZ35oQV.jpg" alt="Original Fundus
256
+ """ , "equation.png",
257
  ],
258
  ],
259
  inputs=[question_input, image_input],