## Inspiration

The inspiration for PillHero was to create a complete solution to helping the sick and elderly to learn and stick too their schedule for taking medication.

## What it does

Using the iOS app, the user enters their medication schedule, which is composed of a start date, end date, daily timings, and the medication name. The user places their medication on the pill mount. As the time nears for the user to take their medication, the app notifies the user that they should take their medication. The pill mount detects when they take their medication. If the user does not, the app notifies them. Should the user not take the pill within a specified period after this, it is counted as them having missed taking their medication.

## How we built it

-- Hardware --
The pill mount was built out of three main components: the Raspberry Pi 3, an infrared LED, and an infrared phototransistor. The LED and phototransistor are used as a touch-less way of detecting when the user removes the pill bottle from the mount. If this was done close enough to when the user is supposed to take their pills, then the assumption is made that the user took their pills for that time slot.
-- Software --
On the software side there was a combination of front end as well as back end developments. For the front end, an iOS mobile application was used to gain input from the user as well as to present calendar information. The input method utilizes speech-to-text technology to translate user voice input to a text input. From there, the user is able to edit the textfield if any mistakes were made. Then to input is passed to an AI for parsing and understanding of the statement. That data is later is sent to a back end server that keeps track of the user's pill intake via communication with the Raspberry Pi 3 and communicating using the calendar screen on the iOS application.

## Challenges we ran into

one of the major challenges we encountered was trying to connect the raspberry pi, the phone and the server all together. Due to the security of the wifi, it was very difficult to ssh into either the raspberry pi or properly set up http requests between the devices. Our work around for this was to set up a LAN that let us connect all our devices and easily make requests between them

## Accomplishments that we're proud of

Completing a project that involves hardware has been a goal of multiple team members for a while and this hackathon gave them the opportunity to do that. In reality though, the team is most proud that they were able to create an internet of things project where the hardware and software complement each other rather than feel arbitrarily attached.

## What we learned

The entire team learned a significant amount about network programming from troubleshooting issues with the Pi 3 and trying to interface the server with both system components. Otherwise, the learning was different for each member. The members who had experience with hardware were exposed to working with servers and databases. Simultaneously, the members with full-stack experience gained exposure to working with hardware. This resulted in all members becoming more well-rounded developers and gaining experience that will aid them in future hackathons and their careers.

## What's next for PillHero

Post-hackathon, the team plans to continue development of PillHero into a more complete product. Numerous upgrades are planned like the ability to track non-daily medication routines and functionality with multiple drugs. On the iOS side in particular, implementation of push notifications and text message reminders are planned. As for the pill mount, the team plans to add an accurate to team plans to add an accurate analog scale to the device so it can notify the user when they are running out of pills.
## Inspiration

Automation is at its peak when it comes to technology, but one area that has lacked to keep up, is areas of daily medicine. We encountered many moments within our family members where they had trouble keeping up with their prescription timelines. In a decade dominated by cell phones, we saw the need to develop something fast and easy, where it wouldn’t require something too complicated to keep track of all their prescriptions and timelines and would be accessibly at their fingertips.

## What it does

CapsuleCalendar is an Android application that lets one take a picture of their prescriptions or pill bottles and have them saved to their calendars (as reminders) based on the recommended intake amounts (on prescriptions). The user will then be notified based on the frequency outlined by the physician on the prescription. The application simply requires taking a picture, its been developed with the user in mind and does not require one to go through the calendar reminder, everything is pre-populated for the user through the optical-character recognition (OCR) processing when they take a snap of their prescription/pill bottle.

## How we built it

The application was built for Android purely in Java, including integration of all APIs and frameworks. First, authorization of individualized accounts was done using Firebase. We implemented and modified Google’s optical-character recognition (OCR) cloud-vision framework, to accurately recognize text on labels, and process and parse it in real-time. The Google Calendar API was then applied on the parsed data, and with further processing, we used intents to set reminders based on the data of the prescriptions labels (e.g. take X tablets X daily - where X was some arbitrary number which was accounted for in a (or multiple) reminders).

## Challenges we ran into

Working with the OCR Java framework was quite difficult to implement into our personalized application due to various dependency failures - it took us way too long to debug and get the framework to work *sufficiently* for our needs. Also, the default OCR graphics toolkit only captures very small snippets of text at a single time whereas we needed multiple lines to be processed at once and text at different areas within the label at once (e.g. default implementation would allow one set to be recognized and processed - we needed multiple sets). The default OCR engine wasn't quite effective for multiple lines of prescriptions, especially when identifying both prescription name and intake procedure - tweaking this was pretty tough. Also, when we tried to use the Google Calendar API, we had extensive issues using Firebase to generate Oauth 2.0 credentials (Google documentation wasn’t too great here :-/).

## Accomplishments that we're proud of

We’re proud of being able to implement a customized Google Cloud Vision based OCR engine and successfully process, parse and post text to the Google Calendar API. We were just really happy we had a functional prototype!

## What we learned

Debugging is a powerful skill we took away from this hackathon - it was pretty rough going through complex, pre-written framework code. We also learned to work with some new Google APIs, and Firebase integrations. Reading documentation is also very important… along with reading lots of StackOverflow.

## What's next for CapsuleCalendar

We would like to use a better, stronger OCR engine that is more accurate at reading labels in a curved manner, and does not get easily flawed from multiple lines of text. Also, we would like to add functionality to parse pre-taken images (if the patient doesn’t have their prescription readily available and only happens to have a picture of their prescription). We would also like to improve the UI.

## Run the application

Simply download/clone the source code from GitHub link provided and run on Android studio. It is required to use a physical Android device as it requires use of the camera - not possible on emulator.
## Inspiration

In our current society, drug abuse and the inability to follow prescribed schedules is a huge problem. To be more specific, every year the United States spends $100-$268 billion on prescription related medical emergencies in addition to the 125,000 annual deaths. This team decided to develop a way to assist individuals in taking their prescribed medication in a more convenient way to target this massive issue.

## What it does

The user is able to text (or say) their schedule for any prescribed medication. Then the device sorts the medicine and dispenses it at the right time. Once dispensed the user also receives a text message reminding them to pick it up. There is also a web app which the user can log in to see their current schedule. The goal is to help those who are unable to keep up with their schedule due to poor time management or simply laziness.

## How we built it

The Web App was developed with NodeJS, HTML, JS, and CSS (bootstrap as well). Firebase was used for our realtime database. Microsoft Azure was used to allow users to simply say their schedule and then it would get analyzed and understood and finally uploaded to the database. Twilio Chatbot was used for more millennial users who would prefer adding or viewing their schedules on their phones. The physical device was made with Arduino and ESP32. Servo's were used to first sort the different types of medication and also for release on the needed time.

## Challenges we ran into

Being able to sort the medication based on its properties, accessing Firebase data on the ESP32, working with Twilio chatbot while making API calls directly from it, and the limited time.

## Accomplishments that we're proud of

That we actually are finished the project in the time period designated for it.

## What we learned

How to use Ngrok, how to use Firebase from the Arduino

## What's next for ReminderX

Look forward to our presentation and you'll find out!