Are you tired of the confusion and uncertainty that often accompanies medication prescriptions? Introducing MedInfoBot, your trusted companion on your healthcare journey. We understand that understanding medication details can be challenging, and that's why we've designed MedInfoBot to simplify medication understanding and make healthcare more accessible than ever before.

## What it does

With MedInfoBot, you can effortlessly extract important information from your prescriptions, including medication names, dosages, and what it does. Say goodbye to the complexity of medical terms. Our user-friendly interface provides clear and concise explanations of common uses of your medicine and potential side effects, along with audio descriptions, for those visually impaired, empowering anyone to make informed decisions about your own health.

## Key Features:

**Prescription Information Processing**
MedInfoBot employs Optical Character Recognition (OCR) to extract prescription details from images, ensuring accurate and reliable data retrieval.

**User-Friendly Interface**
The application boasts an intuitive and user-friendly interface, making it accessible to users of all backgrounds and ages.

**AI-Powered Explanations**
 MedInfoBot utilizes the OpenAI API to generate clear and concise explanations of medication details, enhancing your understanding of what you're taking.

**Hardware Integration**
 Powered by the Milk-V Duo board and the CAM-GC2083 camera module, MedInfoBot leverages efficient hardware capabilities for accurate image capture and processing.

**Expandable Features**
 The Milk-V Duo board's versatility allows for the seamless integration of future additional features, such as human emotion detection and prediction with the camera module.

## Technologies

**Hardware Components**

MedInfoBot operates on a low cost, low power consumption, high performance $5 RISC-V computer, ensuring that our solution is not only accessible but also incredibly efficient.

**Milk-V Duo Board**
 This ultra-compact embedded development platform is powered by the CV1800B chip, capable of running Linux and RTOS. It provides a reliable, low-cost, and high-performance foundation for MedInfoBot. The board features dual processors, versatile GPIO pins, USB support, and more, making it a perfect match for this healthcare application.

**CAM-GC2083 Camera Module**
 The GLAXYCORE's GC2083 CMOS Image Sensor, is a camera module thatoffers up to 2MP resolution, ensuring high-quality imaging. It seamlessly integrates with the 16P MIPI CSI interface on the Milk-V Duo board, enabling advanced imaging capabilities. With features like optical excellence, high sensitivity, and impressive dynamic range, this camera module enhances the MedInfoBot's image capture capabilities significantly.

**Software Components**
This bot makes liberal use of various machine learning/AI models to perform its desired functions.

**PyTesseract**
PyTesseract is a Python wrapper for Google's Tesseract Optical Character Recognition application. This allows the bot to take an image as an input, and output the recognizable text that appears in the image.

**OpenAI/ChatGPT**
OpenAI's ChatGPT LLM was used to process the text output from PyTesseract and extract the medication name from the text. It was also then used to return useful information about the specified medication.

**GTTS**
GTTS (Google Text To Speech) is a python wrapper for Google's Text to Speech functions. It was used to generate an audio file of the medication name and description outputted by the previous software component.

Together, these hardware components and software technologies power MedInfoBot to simplify medication understanding and promote healthcare accessibility.

## What's next for MedInfoBot

As we move forward, MedInfoBot aims to expand its capabilities and impact in the healthcare space. Here's a glimpse of what's on the horizon:

**Multi-Language Support**

Accessibility to healthcare information for a global audience with support for multiple languages.

**Integration with Hume AI**

Exploring emotion detection capabilities to provide personalized support alongside medication information.

**Voice Assistance**

Making MedInfoBot even more accessible with voice command features for medication inquiries.

**User Profiles**

Personalized profiles to track medication history, receive reminders, and get tailored recommendations.

**Medical Database Integration**

Currently, ChatGPT is used to search for information about the given medication. In the future, this can be upgraded to searching in a pre-compiled/pre-assembled medical database in order to speed up bot functions and improve medication accuracy.

MedInfoBot's journey doesn't end here! We are your gateway to simplified medication understanding and informed healthcare decisions. Join us in making healthcare more accessible and understandable for everyone.
## Inspiration

Today, we all are subscribed to a plethora of services. There is also an industry trend. According to a Forbes report, the global subscription e-commerce market is expected to grow from $72.91 billion in 2021 to $120.04 billion in 2022. On average, one in two online platforms you use probably has a subscription option [1]. With so many options and new streaming platforms entering the market, consumers are actively making decisions on which streaming service is the best value proposition for them. A survey conducted in 2021, showed that consumers were excited about new streaming services, such as Disney +. "And nearly two-thirds of consumers intended to terminate or downgrade one or more of their current subscriptions to make room for a new service" [2]. To make matters worse, free trial marketing strategies have exploded since the pandemic, and inflation rates in recent months have spiked to 10% or more in some countries, putting pressure on consumer spending in ways that have been unseen since the 1970s. Streaming giants (such as Netflix) and other subscription services (such as Audible) have hiked their monthly rates in 2023 and with a stressed economy, shown through clear economic trends from high inflation and layoffs in many countries, consumers will want to reevaluate their entertainment options [3]. How do we keep track of all these different subscriptions and how can we evaluate where to cut costs in a currently stressed economy?

In order to help users manage their subscriptions, we decided to build Mindful Subscriptions with the help AI, using Cohere's APIs! With Cohere's API XXX to , we created a classifier to classify bank statement transactions vendors such as from vendors who use a Transaction Model (you pay as you use the products) vs those who use a Subscription Model (a periodic (monthly, yearly, or seasonal) payment to gain access to products or services). We also integrated Cohere's API / SERP API powered contextualized factual bot namely, the Grounded Question Answering bot to offer additional information on existing all Pricing Plans for the services that are users are subscribers for.

## What it does

The Mindful Subscriptions application helps users manage their subscriptions by providing them a list of their active subscriptions and their associated cost. They are also grouped based off of their category such as streaming, food, entertainment, etc.

This project offers users transparency and simplicity into spendings on the various subscription services they are subscribers to. Using our application, they can explore the costs of their paid subscriptions and explore the ways to cut costs. The application dashboard allows users to view the possible the savings they can incur by removing certain subscription from their recurring expenses or by seeing alternative subscription tiers that they could subscribe to instead. On this application, users can freely explore their options on how to meet their needs as well as their financial goals.

Features:

1. Dashboard
	* bankstatement upload: upload bank statement csv
	* subscriptions: add/delete/edit(manage)
2. Toggle Pricing Intel
	* Get more information on the subscription and its pricing plans so that you can re-evaluate the best Pricing Plan to meet your personal needs and financial goals -- Cohere's API / SERP API powered contextualized factual bot namely, the Grounded Question Answering
3. AI Subscription Parser/Classifier
	* AI powered categorization of Transaction Model vs Subscription Model (subscription classification) vendors from historical bank transaction data (bank statement) (Beta)
4. AI Category Classifier:
	* AI powered classifier of the subscription categories (e.g. Education, Entertainment) (Beta)
5. Reminder Feature (Not Implemented)
	* send notifications 3 days in advance of the next renewal date

note: \* beta implies implemented on the backend only

## How we built it

FrontEnd - Add/edit/delete subscriptions

* Reducer Logic to Manage Stateful Front-End Variables such as JSON to turn messy data into interactive and immersive insights
* Engages User to think about the Price of their subscriptions. What are their most expensive subscriptions? What are the cheapest? Which subscriptions sound ideal/frown-creasing according to contextual information gathered by Cohere API? Which subscriptions can become much more expensive than other subscriptions?
* Key-value pair (see example below)
* Parse Bank Statement to populate subscriptions
Backend
* FastAPI
* Cohere (sandbox-grounded-qa <https://github.com/arjunchandra/sandbox-grounded-qa>) - AI subscription classification from historical bank transaction (0/1), AI gathered pricing plan information,
* Categories classifier to classify the subscription categories (e.g. Education, Entertainment)

## Challenges we ran into

* The subscription classification using Cohere ran into headwind during the development as it worked well on a small scale in the playground but not with real bank transaction data when you have 50+ transaction histories.

## Accomplishments that we're proud of

* We successfully integrated Cohere's QA bot in the front-end to check for the price intel

## What we learned

* We learned together about how to make well use of cohere to make AI understand what we truly want based on a bit logics behind the scene. We also realize the potential of AI is infinite and also cohere is such as strong natural language processing model: they can achieve any features as long as we have the ideas and have set everything correctly, which is definitely incredible!
* We just accomplished a project with features we have never thought before as a team! Personally, I think what happened within these approximate 2 days has really blowed my mind. Initially I didn't know anything about frontend, backend, and endpoints, how to connect them together, and I even have never heard of cohere before. But big thanks to my teammates, they explained everything to me very patiently and resolved my concerns without any complains. They also told me to not stress too much about it because everybody starts at 0, and they all learned through a step-by-step process. Really appreciated! I felt like that's the actual significance of the event: not just people getting together, but mind and heart getting together.
* Debugging large language model's performance can be hard. This is why it's important to boardening the thinking when approaching using NLP models for a new use case since they will have their own strength such as content generation, etc.

## What's next for Mindful Subscriptions

* Chatbox for the user to ask more information about the subscription such as steps to unsubscribe from the subscription.
* Price checker: check if there is student discount

Link to the powerpoint: <https://www.hollywoodreporter.com/business/business-news/apple-disney-netflix-price-increases-1235258312/>
[1] <https://www.forbes.com/sites/jiawertz/2022/07/15/the-growth-of-subscription-commerce/?sh=676539a6b572>
[2]<https://koreascience.kr/article/JAKO202119759275785.page>
[3] <https://www.hollywoodreporter.com/business/business-news/apple-disney-netflix-price-increases-1235258312/>
## Inspiration

Imagine you broke your EpiPen but you need it immediately for an allergic reaction. Imagine being lost in the forest with cut wounds and bleeding from a fall but have no first aid kit. How will you take care of your health without nearby hospitals or pharmacies? Well good thing for you, we have **MediFly**!! MediFly is inspired by how emergency vehicles such as ambulances take too long to get to the person in need of aid because of other cars on the road and traffic. Every second spent waiting is risking someone's life. So in order to combat that issue, we use **drones** as the first emergency responders to send medicine to save people's lives or keep them in a stable condition before human responders arrive.

## What it does

MediFly allows the user to request for emergency help or medication such as an Epipen and Epinephrine. First you download the MediFly app and create a personal account. Then you can log into your account and use the features when necessary. If you are in an emergency, press the "EMERGENCY" button and a list of common medication options will appear for the person to pick from. There is also an option to search for your needed medication. Once a choice is selected, the local hospital will see the request and send a drone to deliver the medication to the person. Human first responders will also be called. The drone will have a GPS tracker and a GPS location of the person it needs to send the medication to. When the drone is within close distance to the person, a message is sent to tell them to go outside to where the drone can see the person. The camera will use facial recognition to confirm the person is indeed the registered user who ordered the medication. This level of security is important to ensure that the medication is delivered to the correct person. When the person is confirmed, the medication holding compartment lid is opened so the person can take their medication.

## How we built it

On the software side, the front end of the app was made with react coded in Javascript, and the back end was made with Django in Python.
The text messages work through Twilio. Twilio is used to tell the user that the drone is nearby with the medication ready to hand over. It sends a message telling the person to go outdoors where the drone will be able to find the user.
On the hardware side, there are many different components that make up the drone. There are four motors, four propeller blades, a electronic speed controller, a flight controller, and 3D printed parts such as the camera mount, medication box holder, and some components of the drone frame. Besides this there is also a Raspberry Pi SBC attached to the drone for controlling the on-board systems such as the door to unload the cargo bay and stream the video to a server to process for the face recognition algorithm.

## Challenges we ran into

Building the drone from scratch was a lot harder than we anticipated. There was a lot of setting up that needed to be done for the hardware and the building aspect was not easy. It consisted of a lot of taking apart, rebuilding, soldering, cutting, hot gluing, and rebuilding.

Some of the video streaming systems did not work well at first, due to the CORS blocking the requests, given that we were using two different computers to run two different servers.

Traditional geolocation techniques often take too long - as such, we needed to build a scheme to cache a user's location before they decided to send a request to prevent lag. Additionally, the number of pages required to build, stylize, and connect together made building the site a notable challenge of scale.

## Accomplishments that we're proud of

We are extremely proud of the way the drone works and how it's able to move at quick, steady speeds while carrying the medication compartment and battery.
On the software side, we are super proud of the facial recognition code and how it's able to tell the difference between different peoples' faces. The front and back end of the website/app is also really well done. We first made the front end UI design on Figma and then implemented the design on our final website.

## What we learned

For software we learned how to use React, as well as various user authorization and authentication techniques. We also learned how to use Django.
We learnt how to build an accurate, efficient and resilient face detection recognition and tracking system to make sure the package is always delivered to the correct person.
We experimented with and learned various ways to stream real-time video over a network, also over longer ranges for the drone.
For hardware we learned how to set up and construct a drone from scratch!

## What's next for MediFly

In the future we hope to add a GPS tracker to the drone so that the person who orders the medication can see where the drone is on its path.
We would also add Twilio text messages so that when the drone is within a close radius to the user, it will send a message notifying the person to go outside and wait for the drone to deliver the medication.