## Inspiration

A couple weeks ago, a friend was hospitalized for taking Advil–she accidentally took 27 pills, which is nearly 5 times the maximum daily amount. Apparently, when asked why, she responded that thats just what she always had done and how her parents have told her to take Advil. The maximum Advil you are supposed to take is 6 per day, before it becomes a hazard to your stomach.

#### PillAR is your personal augmented reality pill/medicine tracker.

It can be difficult to remember when to take your medications, especially when there are countless different restrictions for each different medicine. For people that depend on their medication to live normally, remembering and knowing when it is okay to take their medication is a difficult challenge. Many drugs have very specific restrictions (eg. no more than one pill every 8 hours, 3 max per day, take with food or water), which can be hard to keep track of. PillAR helps you keep track of when you take your medicine and how much you take to keep you safe by not over or under dosing.

We also saw a need for a medicine tracker due to the aging population and the number of people who have many different medications that they need to take. According to health studies in the U.S., 23.1% of people take three or more medications in a 30 day period and 11.9% take 5 or more. That is over 75 million U.S. citizens that could use PillAR to keep track of their numerous medicines.

## How we built it

We created an iOS app in Swift using ARKit. We collect data on the pill bottles from the iphone camera and passed it to the Google Vision API. From there we receive the name of drug, which our app then forwards to a Python web scraping backend that we built. This web scraper collects usage and administration information for the medications we examine, since this information is not available in any accessible api or queryable database. We then use this information in the app to keep track of pill usage and power the core functionality of the app.

## Accomplishments that we're proud of

This is our first time creating an app using Apple's ARKit. We also did a lot of research to find a suitable website to scrape medication dosage information from and then had to process that information to make it easier to understand.

## What's next for PillAR

In the future, we hope to be able to get more accurate medication information for each specific bottle (such as pill size). We would like to improve the bottle recognition capabilities, by maybe writing our own classifiers or training a data set. We would also like to add features like notifications to remind you of good times to take pills to keep you even healthier.
## Inspiration

Automation is at its peak when it comes to technology, but one area that has lacked to keep up, is areas of daily medicine. We encountered many moments within our family members where they had trouble keeping up with their prescription timelines. In a decade dominated by cell phones, we saw the need to develop something fast and easy, where it wouldn’t require something too complicated to keep track of all their prescriptions and timelines and would be accessibly at their fingertips.

## What it does

CapsuleCalendar is an Android application that lets one take a picture of their prescriptions or pill bottles and have them saved to their calendars (as reminders) based on the recommended intake amounts (on prescriptions). The user will then be notified based on the frequency outlined by the physician on the prescription. The application simply requires taking a picture, its been developed with the user in mind and does not require one to go through the calendar reminder, everything is pre-populated for the user through the optical-character recognition (OCR) processing when they take a snap of their prescription/pill bottle.

## How we built it

The application was built for Android purely in Java, including integration of all APIs and frameworks. First, authorization of individualized accounts was done using Firebase. We implemented and modified Google’s optical-character recognition (OCR) cloud-vision framework, to accurately recognize text on labels, and process and parse it in real-time. The Google Calendar API was then applied on the parsed data, and with further processing, we used intents to set reminders based on the data of the prescriptions labels (e.g. take X tablets X daily - where X was some arbitrary number which was accounted for in a (or multiple) reminders).

## Challenges we ran into

Working with the OCR Java framework was quite difficult to implement into our personalized application due to various dependency failures - it took us way too long to debug and get the framework to work *sufficiently* for our needs. Also, the default OCR graphics toolkit only captures very small snippets of text at a single time whereas we needed multiple lines to be processed at once and text at different areas within the label at once (e.g. default implementation would allow one set to be recognized and processed - we needed multiple sets). The default OCR engine wasn't quite effective for multiple lines of prescriptions, especially when identifying both prescription name and intake procedure - tweaking this was pretty tough. Also, when we tried to use the Google Calendar API, we had extensive issues using Firebase to generate Oauth 2.0 credentials (Google documentation wasn’t too great here :-/).

## Accomplishments that we're proud of

We’re proud of being able to implement a customized Google Cloud Vision based OCR engine and successfully process, parse and post text to the Google Calendar API. We were just really happy we had a functional prototype!

## What we learned

Debugging is a powerful skill we took away from this hackathon - it was pretty rough going through complex, pre-written framework code. We also learned to work with some new Google APIs, and Firebase integrations. Reading documentation is also very important… along with reading lots of StackOverflow.

## What's next for CapsuleCalendar

We would like to use a better, stronger OCR engine that is more accurate at reading labels in a curved manner, and does not get easily flawed from multiple lines of text. Also, we would like to add functionality to parse pre-taken images (if the patient doesn’t have their prescription readily available and only happens to have a picture of their prescription). We would also like to improve the UI.

## Run the application

Simply download/clone the source code from GitHub link provided and run on Android studio. It is required to use a physical Android device as it requires use of the camera - not possible on emulator.
## 🎇 Inspiration 🎇

Since the last few years have been full of diseases, illness, a huge number of people have been in need of medical assistance. Our elders suffer from certain illnesses and have a hard time remembering which medications to take, in what amount and what time. So, Similarly We made an app that saves all medical records and prescription either in email form, scanned copy or written form and setting medicine reminders that enables client to determine the pill amount and timings to take pills for everyday.

## ❓ What it does ❓

* Problem 1 - Most of the patients frequently have difficulty in remembering or managing multiple medications .Many a times they forget medicine name and how they look like, especially for elders. Medication non-adherence leads to 125,000 preventable deaths each year, and about $300 billion in avoidable healthcare costs.
* Problem 2 - We all have multiple folders in back of our closets full of medical records, prescriptions , test reports etc. which lets all agree are a hassle to carry everytime a doctor asks for our medical history .
* Problem 3 - In times of emergency every minute counts and in that minute finding contacts of our caregivers, nearby hospitals, pharmacies and doctors is extremely difficult.

## 🤷‍♂️ How we built it 🤷‍♂️

We built it with technologies like Firebase, flutter for providing better contents and SQL lite for storage.

## 🤔 Challenges we ran into 🤔

* CRUD operations in firebase took a lot more of our time than expected.
* Pushing image notifications was another obstacle but with constant Hard work and support we finally completed it .
* Handling this project with the college classes was a test of our time management and that's totally worth it at end.

## 👏 What we learned 👏

* We learned how to worked quickly with Dart along with that we also learned how to use packages like contact picker, airplane mode checker in a effective way.
* It was my [ Aditya Singh ] second time using Swift and I had a lot of fun and learnings while using it.
* Design all the logo's as well as the UI part was handled by me [ Sanyam Saini ] , I learned many new features in Figma that I didn't knew before.

## 🚀 What's next for Med-Zone 🚀

* In the future, we can add additional features like providing diet charts, health tips, exercises / Yogas to follow for better health and nutrition, and some expert insights as well for which we can charge the subscription fee.
* We can add different types of tunes/ringtones according to which medications to take at which time.