Spaces:
Running
on
Zero
Running
on
Zero
Update app.py
Browse files
app.py
CHANGED
@@ -63,7 +63,7 @@ def multimodal_understanding(image, question, seed, top_p, temperature):
|
|
63 |
pad_token_id=tokenizer.eos_token_id,
|
64 |
bos_token_id=tokenizer.bos_token_id,
|
65 |
eos_token_id=tokenizer.eos_token_id,
|
66 |
-
max_new_tokens=
|
67 |
do_sample=False if temperature == 0 else True,
|
68 |
use_cache=True,
|
69 |
temperature=temperature,
|
@@ -193,8 +193,67 @@ with gr.Blocks() as demo:
|
|
193 |
"doge.png",
|
194 |
],
|
195 |
[
|
196 |
-
"
|
197 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
198 |
],
|
199 |
],
|
200 |
inputs=[question_input, image_input],
|
|
|
63 |
pad_token_id=tokenizer.eos_token_id,
|
64 |
bos_token_id=tokenizer.bos_token_id,
|
65 |
eos_token_id=tokenizer.eos_token_id,
|
66 |
+
max_new_tokens=4000,
|
67 |
do_sample=False if temperature == 0 else True,
|
68 |
use_cache=True,
|
69 |
temperature=temperature,
|
|
|
193 |
"doge.png",
|
194 |
],
|
195 |
[
|
196 |
+
"""Analyze the provided fundus image in exhaustive detail, following the standard ophthalmological protocol for fundus examination. Output an HTML report structured as a formal medical document. The report MUST:
|
197 |
+
|
198 |
+
1. **Image Quality Assessment:** Begin with a concise assessment of image quality, noting focus, illumination, field of view, and any artifacts (and their impact on assessability).
|
199 |
+
|
200 |
+
2. **Detailed Clinical Findings:** Describe each of the following areas with the utmost precision and specificity, using proper ophthalmological terminology:
|
201 |
+
* **Optic Disc:**
|
202 |
+
* Size and shape (including any abnormalities).
|
203 |
+
* Color (specifically noting any pallor and its location).
|
204 |
+
* Cup-to-Disc Ratio (CDR), providing both vertical and horizontal estimates.
|
205 |
+
* Neuroretinal Rim: Assess rim thickness in all quadrants (superior, inferior, nasal, temporal). Explicitly state whether the ISNT rule is followed or violated. Describe any notching or focal thinning.
|
206 |
+
* Peripapillary Region: Describe the presence/absence of peripapillary atrophy (PPA), differentiating between alpha and beta zones. Note any hemorrhages.
|
207 |
+
* **Retinal Vasculature:**
|
208 |
+
* Arterioles: Describe caliber (narrowing, dilation), tortuosity, and any focal abnormalities.
|
209 |
+
* Venules: Describe caliber, tortuosity, and any abnormalities.
|
210 |
+
* Arteriovenous (A/V) Ratio: Estimate the A/V ratio.
|
211 |
+
* Crossing Changes: Note any arteriovenous nicking or other crossing abnormalities.
|
212 |
+
* Vessel Course: Describe the course of the major vessels, and check for abnormalities.
|
213 |
+
* **Macula:**
|
214 |
+
* Foveal Reflex: Describe the presence/absence and quality of the foveal reflex.
|
215 |
+
* Pigment Changes: Note any pigmentary abnormalities, drusen, or other lesions.
|
216 |
+
* Edema/Exudates: Describe any signs of macular edema or exudates.
|
217 |
+
* **Peripheral Retina:**
|
218 |
+
* Mid-Periphery: Describe any abnormalities (hemorrhages, exudates, tears, etc.).
|
219 |
+
* Far Periphery: Note the extent of visualization and any findings.
|
220 |
+
|
221 |
+
3. **Differential Diagnosis:** Based solely on the image findings, provide a prioritized differential diagnosis. Include the most likely diagnosis and any other plausible possibilities. For each diagnosis, explain the reasoning based on the observed features.
|
222 |
+
|
223 |
+
4. **Diagnostic Confidence:** Indicate the confidence level for the primary diagnosis. List the key image findings that support the diagnosis.
|
224 |
+
|
225 |
+
5. **Simulated AI Attention Metrics:** Create a table representing a *simulated* AI attention distribution. This should reflect the expected focus areas for the most likely diagnosis, based on the known importance of different features. Provide percentages for:
|
226 |
+
* Optic Disc (Total)
|
227 |
+
* Cup
|
228 |
+
* Neuroretinal Rim (subdivided by region if significant differences exist)
|
229 |
+
* Peripapillary Atrophy
|
230 |
+
* Vessels
|
231 |
+
* Macula
|
232 |
+
* Periphery
|
233 |
+
|
234 |
+
6. **Summary and Impression:** Provide a concise summary of the key findings and the overall impression.
|
235 |
+
|
236 |
+
7. **Recommendations:**
|
237 |
+
* Provide specific, actionable recommendations based on the image findings.
|
238 |
+
* If referral is warranted, clearly state the urgency and the type of specialist.
|
239 |
+
* List any recommended investigations (e.g., OCT, visual fields).
|
240 |
+
|
241 |
+
8. **Disclaimer:** Include a disclaimer stating that the report is based on image analysis alone and does not replace a full clinical examination.
|
242 |
+
|
243 |
+
9. **HTML Structure:** Use semantic HTML elements (h1-h3, p, ol, ul, table, div) to create a well-structured, readable report. Include:
|
244 |
+
* A report header with a title ("EyeUnit.ai | AI for Ophthalmology") and a logo placeholder.
|
245 |
+
* An image comparison section displaying the original fundus image and a placeholder for a heatmap (a canvas element with id "heatmapCanvas"). No actual heatmap generation is required; the canvas is a placeholder.
|
246 |
+
* A placeholder for patient information(PATIENT ID, NAME, AGE, DATE OF EXAM)
|
247 |
+
* Clearly labeled sections for each part of the analysis.
|
248 |
+
* Tables for the "Overall Analysis Coverage" and "AI-Driven Attention Metrics."
|
249 |
+
|
250 |
+
10. **CSS Styling:** Apply CSS styles to make the report visually appealing and professional. The report should be suitable for both screen viewing and printing (use a `@media print` block to optimize for print).
|
251 |
+
* **Crucial Details:**
|
252 |
+
* **PATIENT ID, NAME, AGE and DATE OF EXAM**
|
253 |
+
|
254 |
+
11. **Crucial Details:** Output ONLY the complete HTML code. Do not provide any surrounding text or explanations. Focus solely on generating the HTML report.
|
255 |
+
12. **IMG SOURCE:** Use this image as the image source: `<img src="https://i.imgur.com/kZ35oQV.jpg" alt="Original Fundus
|
256 |
+
""" , "equation.png",
|
257 |
],
|
258 |
],
|
259 |
inputs=[question_input, image_input],
|