## Inspiration

The inspiration behind Shifa comes from the desire to simplify medication management for the elderly and individuals who may struggle with deciphering medical labels and understanding complex prescription instructions. We recognized the need to leverage technology to improve medication adherence and enhance understanding of medication details. The name Al Shifa comes from the arabic word healing or remedy and it captures our desire to push a new wave of technological advancement towards prioritizing the health and healing of individuals.

## What it does

Shifa uses Optical Character Recognition (OCR) technology to extract text from prescription labels and medical pill bottles. It then utilizes the OpenAI API to generate simple and straightforward descriptions of the medication and its uses. This means that Shifa takes the complicated jargon often found on medication labels and translates it into easily understandable language.

The primary function of Shifa is to help the elderly and anyone managing multiple medications to stay organized and informed about their treatment. It simplifies the process of medication management, ensuring that users have a clear understanding of what each medication is for and how to take it.

## How we built it

* **OCR**: We used image preprocessing and the OCR package Tesseract to scan and extract text from images of medical pill bottles and prescription labels.
* **OpenAI API**: We prompt-engineered the davinci model from the OpenAI API to parse through the extracted OCR text and generate simplified descriptions of the medication and its uses.
* **React**: We used React to create an intuitive and user-friendly frontend interface. This allows users to easily interact with the application and receive the information they need.
* **Flask**: Flask was employed to link the OCR and OpenAI backend with the React frontend, ensuring seamless communication and functionality.

## Challenges we ran into

* **OCR Accuracy**: Ensuring accurate text extraction from images posed a significant challenge, as prescription labels and pill bottles can vary in appearance and quality.
* **Integration**: Integrating multiple technologies and ensuring they work together smoothly required careful planning and testing.

## Accomplishments that we're proud of

* Developing a robust davinci model through prompt-engineering and parsing algorithms that can convert complex medical information into user-friendly language.
* Creating an easy-to-use and visually appealing frontend with React.
* Building a functional and reliable application that has the potential to significantly improve medication management for the elderly.

## What we learned

* The power of NLP in making complex medical information accessible to a wider audience.
* The importance of user-friendly design and interface in healthcare applications.
* The potential impact of technology in improving the lives of seniors and individuals with complex medication regimens.

## What's next for Shifa

The future of Shifa holds exciting possibilities, and we have designed a Figma demo to present these future enhancements.

* **Add Audio**: We hope to add audio corresponding to each scan to increase the accessibility of our app.
* **Enhanced Accuracy**: We aim to improve the accuracy of text extraction from images through further refinement of the OCR technology.
* **Expanded Medication Database**: Shifa can be enhanced by incorporating a comprehensive database of medications, providing users with more detailed information about their prescriptions.
* **Personalized Reminders**: Implementing medication reminder features to help users stay on track with their treatment plans.
* **Integration with Healthcare Providers**: Exploring options to connect Shifa with healthcare providers to ensure seamless communication and medication management.
* **Mobile App**: Developing a mobile version of Shifa for even greater accessibility and convenience.
* **Medication Scheduling**: Introducing a calendar feature that allows users to schedule when medications need to be taken. This feature will provide timely reminders, ensuring that users never miss a dose and helping them adhere to their treatment plans more effectively.
## Inspiration

The idea for IngredientAI came from my personal frustration with navigating complex and often misleading ingredient labels on beauty and personal care products. I realized that many consumers struggle to understand what these ingredients actually do and whether they are safe. The lack of accessible information often leaves people in the dark about what they are using daily. I wanted to create a tool that brings transparency to this process, empowering users to make healthier and more informed choices about the products they use.

## What it does

The idea for IngredientAI was born out of the frustration of navigating complex and often misleading ingredient labels on beauty and personal care products. I realized that many consumers struggle to understand what these ingredients actually do and whether they are safe. The lack of accessible information often leaves people in the dark about what they are using daily. I wanted to create a tool that brings transparency to this process, empowering users to make healthier and more informed choices about the products they use.

## How I built It

The frontend is built using React Native and ran using Expo. Users interact with a FlatList component that accesses a backend database powered by Convex. Text extracted from images as well as generates ingredient descriptions is all done through OpenAI's gpt-4o-mini large-language model.

## Challenges I ran into

A big challenge that I come across was figuring out how to extract text from images. Originally, I planned on setting up a server-side script that makes use of Tesseract.js's OCR capabilities. However, after some testing, I realized that I did not have enough time to fine tune Tesseract so that it extracts text from images under a variety of different lighting. For IngredientAI to be used by consumers, it must be able to work under a wide variety of circumstances. To solve this issue, I decided it would be best for me to use OpenAI's new Vision capabilities. I did not go with this originally because I wanted to minimize the amount of OpenAI API calls I made. However, under time constraints, this was the best option.

## Accomplishments that I'm proud of

I am extremely proud of how far my App Development has come. At a previous hackathon in March, I had used React Native for the very first time. At that hackathon, I was completely clueless with the technology. A lot of my code was copy/pasted from ChatGPT and I did not have a proper understanding of how it worked. Now, this weekend, I was able to create a fully functional mobile application that has organized (enough) code that allows me to expand on this project in the future.

## What I learned

Every hackathon, my goal is to learn at least one new technology. This weekend, I decided to use Convex for the very first time. I really appreciated the amount of resources that Convex provides for learning their technology. It was especially convenient that they had a dedicated page for hackathon projects. It made setting up my database extremely fast and convenient, and as we know, speed is key in a hackathon.

## What's next for IngredientAI

My aim is to eventually bring IngredientAI to app stores. This is an app I would personally find use for, and I would like to share that with others. Future improvements and features include:

* Categorization and visualization of ingredient data
* Suggested products section
* One-button multi-store checkout

I hope you all get the chance to try out IngredientAI in the near future!
## Inspiration

The opioid crisis is a widespread danger, affecting millions of Americans every year. In 2016 alone, 2.1 million people had an opioid use disorder, resulting in over 40,000 deaths. After researching what had been done to tackle this problem, we came upon many pill dispensers currently on the market. However, we failed to see how they addressed the core of the problem - most were simply reminder systems with no way to regulate the quantity of medication being taken, ineffective to prevent drug overdose. As for the secure solutions, they cost somewhere between $200 to $600, well out of most people’s price ranges. Thus, we set out to prototype our own secure, simple, affordable, and end-to-end pipeline to address this problem, developing a robust medication reminder and dispensing system that not only makes it easy to follow the doctor’s orders, but also difficult to disobey.

## What it does

This product has three components: the web app, the mobile app, and the physical device. The web end is built for doctors to register patients, easily schedule dates and timing for their medications, and specify the medication name and dosage. Any changes the doctor makes are automatically synced with the patient’s mobile app. Through the app, patients can view their prescriptions and contact their doctor with the touch of one button, and they are instantly notified when they are due for prescriptions. Once they click on on an unlocked medication, the app communicates with LocPill to dispense the precise dosage. LocPill uses a system of gears and motors to do so, and it remains locked to prevent the patient from attempting to open the box to gain access to more medication than in the dosage; however, doctors and pharmacists will be able to open the box.

## How we built it

The LocPill prototype was designed on Rhino and 3-D printed. Each of the gears in the system was laser cut, and the gears were connected to a servo that was controlled by an Adafruit Bluefruit BLE Arduino programmed in C.

The web end was coded in HTML, CSS, Javascript, and PHP. The iOS app was coded in Swift using Xcode with mainly the UIKit framework with the help of the LBTA cocoa pod. Both front ends were supported using a Firebase backend database and email:password authentication.

## Challenges we ran into

Nothing is gained without a challenge; many of the skills this project required were things we had little to no experience with. From the modeling in the RP lab to the back end communication between our website and app, everything was a new challenge with a lesson to be gained from. During the final hours of the last day, while assembling our final product, we mistakenly positioned a gear in the incorrect area. Unfortunately, by the time we realized this, the super glue holding the gear in place had dried. Hence began our 4am trip to Fresh Grocer Sunday morning to acquire acetone, an active ingredient in nail polish remover. Although we returned drenched and shivering after running back in shorts and flip-flops during a storm, the satisfaction we felt upon seeing our final project correctly assembled was unmatched.

## Accomplishments that we're proud of

Our team is most proud of successfully creating and prototyping an object with the potential for positive social impact. Within a very short time, we accomplished much of our ambitious goal: to build a project that spanned 4 platforms over the course of two days: two front ends (mobile and web), a backend, and a physical mechanism. In terms of just codebase, the iOS app has over 2600 lines of code, and in total, we assembled around 5k lines of code. We completed and printed a prototype of our design and tested it with actual motors, confirming that our design’s specs were accurate as per the initial model.

## What we learned

Working on LocPill at PennApps gave us a unique chance to learn by doing. Laser cutting, Solidworks Design, 3D printing, setting up Arduino/iOS bluetooth connections, Arduino coding, database matching between front ends: these are just the tip of the iceberg in terms of the skills we picked up during the last 36 hours by diving into challenges rather than relying on a textbook or being formally taught concepts. While the skills we picked up were extremely valuable, our ultimate takeaway from this project is the confidence that we could pave the path in front of us even if we couldn’t always see the light ahead.

## What's next for LocPill

While we built a successful prototype during PennApps, we hope to formalize our design further before taking the idea to the Rothberg Catalyzer in October, where we plan to launch this product. During the first half of 2019, we plan to submit this product at more entrepreneurship competitions and reach out to healthcare organizations. During the second half of 2019, we plan to raise VC funding and acquire our first deals with healthcare providers. In short, this idea only begins at PennApps; it has a long future ahead of it.