## Inspiration

We were inspired by the development of foldscope, a very low-cost, high-resolution foldable microscope (<https://en.wikipedia.org/wiki/Foldscope>) capable of imaging blood cells. We wanted to create tools that can integrate with this microscope and, more generally, other applications and better improve health in the developing world. Malaria is a leading cause of death in many developing countries, where blood smears are used to identify the presence of parasites in red blood cells (RBCs). To improve the efficiency of detecting parasitic cells in blood smears, ultimately speeding up malaria diagnosis, we aimed to create an online tool that can do this in minutes.

## What it does

The project allows a user to upload a thin blood smear image, and it classifies the RBCs in the blood that are infected with malaria parasites.

## How we built it

We utilized a thin blood smear dataset from 193 patients in a Bangladesh hospital curated by the NIH. It consists of 20,000 labeled cells (exhibiting malaria parasitic infection or not) across 965 blood smear images. Given these images, we performed a multi-step image segmentation procedure to isolate the red blood cells (RBCs): we first used U-Net to segment the blood smears into cell clusters, then used Faster R-CNN to segment the cell clusters into individual RBCs, and then incorporated thresholding techniques to refine the segmentation and smooth the edges. Once each RBC in every blood smear was individually segmented, we trained a CNN to classify whether these segmented images contained the malaria parasite. Mapping these now-labeled segmented images back to their parent blood smears allowed us to output modified blood smear images highlighting the RBCs containing a malaria parasite.

## Challenges we ran into

We encountered challenges in building an effective RBC segmentation pipeline. Variations in the segmentation procedure greatly affected the classification performance of the CNN, which was somewhat surprising. The various segmentation methodologies we explored yielded segments that looked visually very similar to hand-drawn segmentations provided in the NIH dataset, and these hand-drawn segmentations were classified very well by CNN. We tried integrating various thresholding, grayscale manipulations, filtering, and flood-filling methodologies to integrate with the U-Net + R-CNN for RBC segmentation. In addition, we originally started with pre-trained models like ResNet-18 for classification. However, they tended to overfit the training data, so we opted for a simple, untrained one-layer CNN architecture, which worked the best.

## Accomplishments that we're proud of

We are proud that we were able to build a comprehensive segmentation and classification pipeline and that we were able to integrate this into a full-stack web app with a front end and a back end.

## What we learned

We learned many technical skills along the way, such as using Python’s OpenCV framework for image processing/manipulation, various image segmentation methodologies, and using flask to build out the web app.

## What's next for Plasmodium

In the future, we hope to continue developing our app and refining our segmentation/classification methodologies to increase accuracy. Furthermore, we plan to expand our pipeline to other diseases, such as sickle cell anemia, to create a more comprehensive health diagnostic tool for the developing world.
## Inspiration

Every year, the amount of data collected exponentially grows. As the abundance of data grows, so do the possibilities that come along with it. In conjunction with machine learning in Python, we decided to utilize the tools available to try to improve a critical aspect of the health industry: cancer diagnosis.

## What it does

Our algorithm diagnoses the patient, given traits about their biopsy lab results. With the data of breast cancer on a cellular level, we were able to train a learning algorithm to predict an accuracy of 99% on our test set. In an effort to decrease the amount of false negative diagnosis on our algorithm's behalf, we were able to achieve a 0.4% false negative diagnosis.

## How we built it

In terms of data, we accessed the breast cancer dataset from UCI's machine learning repository. Once we had the data, we used Python and various packages within Python to both clean up and visualize our data. We then used Tensorflow to model this data using 3 different machine learning algorithms: logistic regression, softmax regression, and neural networks. Using a 60% / 40% data split of our data, we trained and tested our models.

## Challenges we ran into

The breast cancer dataset that we used contained only 539 incidences. At the beginning, we had hoped for larger datasets that could train a more sophisticated model. As a result, we had to make do with a smaller model, but still managed to achieve great results.

## Accomplishments that we're proud of

Both Tate and I are incredibly proud of ourselves for coming this far in all. This is both of our first hackathons where we submitted our projects. Furthermore, neither of us had attempted a project in this field in the past, and found that our respective knowledges in machine learning and Tensorflow piggybacked off of each other and pushed ourselves to a newer level.

## What we learned

Throughout Treehacks, we experienced the effects of extreme sleep deprivation, poor diet, and high strain. We vow to pack acai bowls to the next hackathon we go to along with an air mattress. Jokes aside, we threw ourselves into the water with analyzing and modeling learning algorithms in tensorflow as we had little prior experience beforehand. We also went above the typical matplotlib in Python for visuals and experimented with Seaborn for next level visualizations.

## What's next for Breast Cancer Classifier

We look to expand to bigger datasets
## Inspiration

dermalab was inspired by a very common problem that doctors face: disease misdiagnoses. Skin diseases are often misdiagnosed by dermatologists because conditions often contain many similarities in appearance. Doctors - or even patients at times -, unaided with advanced technology, are often less sure of skin disease diagnoses, which may result in incorrect treatment, or even failure of treatment altogether. Skin disease misdiagnosis is primarily relevant in low-income communities where there is a lack of medical doctors with the appropriate specialty and expertise.

Machine learning (ML) and Artificial Intelligence (AI) technologies can be used to detect differences more accurately than the human eye but currently aren’t widely adopted due to a lack of algorithms that are able to conduct a variety of tasks for disease diagnosis. Current algorithms are specialized to a single task in this journey and are often difficult to use, for both patients and doctors.

Therefore, we strove to create an algorithm that was able to successfully leverage these technologies while catering to the entirety of the disease diagnosis process. We wanted to enable doctors to have more confidence in their diagnoses as well as provide helpful, understandable explanations to patients during this journey. dermalab is what we've produced to solve these issues, and we hope you enjoy it!

## What it does

dermalab helps fill the aforementioned gaps via real-time skin condition classification, disease severity calculation, and spread predictions using cutting-edge technology. We've consolidated all of our amazing features on a website that anybody can use!

Users first have an option of uploading pictures of the affected area to the website. Our machine learning algorithm will then classify and identify the disease or condition that is present, if any. However, we don't stop there! Users also receive a summary of their conditions - without all of the hard-to-understand medical jargon. That way, both patients and doctors are on the same page when it comes to understanding a disease diagnosis.

Our web page also gives users the choice of completing a questionnaire to return the severity level of the disease. We provide explanations for each severity level to ensure that this process is empathetic to patients in the best way possible. Our deep learning algorithm also outputs the predicted disease spread for doctors as well. This was done via further analysis through deep learning and MATLAB to understand region, rate, and direction of disease growth. We used the InceptionResNetV2 model, with lesion images and their annotated masks to train the deep learning model, to perform semantic segmentation on unseen data. Once the semantic segmentation of lesions are done, thereby performing binary masks, MATLAB could be used to perform an onion ring segmentation of the region, where dilation and erosion of mask is done and the difference is taken.

Combined, the variety of services that dermalab provides a robust interface for doctors and patients during disease diagnoses, and helps prevents misdiagnoses.

Check it out [here](https://dermalabs-pennapps.co/)!

## How we built it

dermalab can be primarily be split into the frontend and backend. The frontend consists of the web application and the backend contains the ML/AI/deep learning/LLM models.

The backend has four objectives:
A. Diagnosis: Classification of skin diseases
B. Evaluation: Calculation of disease severity level
C. Prognosis: Prediction of disease spread
D. Explanation: Understandable relaying of information to patients

A. Diagnosis: Classification of skin diseases
dermalab uses machine learning and image processing to identify the type of disease that a person is suffering from.

* We trained a random forest model on a dataset consisting of multiple labeled types of skin diseases
* Weights of trained model were saved and loaded with the user-provided image containing affected region.
* Model returns the class the image falls under,
* Success! Identification of the diseases through the image provided is outputted to the user.

B. Evaluation: Calculation of disease severity level
While it is certainly helpful to identify the diseases the person might have, the severity of the diseases plays a key role in determining what measures to take next.

* Users are asked to give their medical history and the severity level of their disease is given.
* Random Forest model is trained with cross validation + medical history dataset of patients with Eryhemato-Squamous dataset
* Results vary from 0 to 5, where 0 indicates the absence of disease and 5 indicates the greatest severity.

C. Prognosis: Prediction of disease spread
dermalab also predicts disease progression.

* Further analysis via deep learning and MATLAB to understand region, rate, and direction of disease growth.
* InceptionResNetV2 model with lesion images and annotated masks for semantic segmentation
* Onion ring segmentation of affected region performed via MATLAB
* More informed doctors making better decisions for treatment plans

D. Explanation: Understandable relaying of information to patients
dermalab supports powerful physician-patient interactions.
When it comes to disease classification, the Metaphor API returns up-to-date information about disease, overcoming a limitation of ChatGPT.
In terms of disease severity (0 to 5), Llama 2.0 and Replicate provide explanation of severity, allowing us to show that LLMs can be used for language simplification.

**Other Tools!**

*MATLAB*
dermalab uses MATLAB as a major tool to analyse the spread of lesions in skin: semantic segmentation further segmented to onion ring segmentations, providing information about origination, growth rate, and direction.

*Metaphor*
We use Metaphor's API because we believe that medical communication requires the most recent research. Patients have a right to access the most recent information available, and Metaphor's API is the perfect solution.

The frontend was connected using Flask and Github.

## Challenges we ran into

One challenge we ran into was running the Metaphor API. It took a lot of troubleshooting, debugging, and asking for help to resolve the issue, but we're glad we did!

Another challenge was the lack of annotated dermatological data. Since we aren't medical professionals that are qualified to annotate available data, we had to get creative when it came to finding data that could achieve our objectives.

## Accomplishments that we're proud of

We're proud, first and foremost, that we were able to bring dermalab to completion.
One accomplishment was linking the frontend (web page) and the backend (multiple ML models) to create a seamless interface for the user.
Another major accomplishment was troubleshooting with Metaphor's API until it was able to output the results we wanted.

## What we learned

We're so grateful for everything that we learned!

* Combining frontend and backend via Flask
* Using Metaphor's API to return improved responses from ChatGPT
* Employing machine learning, deep learning, and MATLAB for dermatology applications

## What's next for dermalab

We want dermalab to be a tool for all doctors and patients. To get there, we want to refine our algorithm to be able to take into account more disease classes, so that it can classify a greater variety of diseases.