## Inspiration

In November 2011, I (Akshay Shankar) suffered from an Avulsion fracture. My thigh ripped the corner of my hip apart and surgery was a must. Post surgery, my doctor didn't have the time to help me with personal exercises and I was financially incapable of affording a personal physical therapist. Faced by this dilemma I did everything from attempting to go advice from my own doctor, to my own intuition and the internet. Finally, I gave up on physical therapy altogether due to the lack of a credible source of training. Now, I face a blunt ache in my hip that I may have to live with forever.

Just like the account above, the smartPhy team members have heard many friends of ours who have suffered Anterior Cruciate Ligament (ACL) tears, dislocations, etc. not pursuing long term physical therapy due to the high cost and time intensive nature of the care provided. Appointments need to be made sporadically over a long period of time, which some people can't afford. Meanwhile, people lack the motivation to perform recommended exercises themselves due to fear of incorrect form which may lead to permanent damage to the affected area.

Our team believed that we have the technology and resources available to bridge the gap between patients and physical therapists. **This was our motivation for smartPhy: to give all patients the equal opportunity to reliable physical therapy for their future quality of life.**

## What it does

SmartPhy is a wearable that performs real time control and monitoring of the physiotherapy routine of the patient. It has two separate components i.e. the wearable and a mobile application.

The mobile application is written for Android and acts as the first user interface for the patient. On the app, the patient can view his/her exercises (with video instructions) which are specific to the patient's post surgery physical condition and are recommended by the doctor. The patient can use the app to select a workout, switch and stop a workout at any time. At the end of a workout, the user can view details of the workout on the app like number of reps logged, total time taken etc. as well as the ability to notify the doctor. The doctor can use the provided information to have a reliable record of their patient's progress not solely based of their patient's account or memory.

When the user starts a workout on the app, it transfers control to the wearable to monitor the workout. In addition, there is a transfer of data (initial and final configurations of the workout) as well which is performed over Bluetooth. Much like a physical instructor, it provides feedback to the patient to ensure correct execution of the exercises. In its current form, smartPhy is a knee brace that with a mounted accelerometer such that it gives the angle of the patient's calf with respect to the knee. The two kinds of feedback in the SmartPhy are haptic (using a vibration motor) and visual (using an LED) which work together to notify the real time state of the patient's routine. All of data processing occurs on a micro-controller and the results are sent back to the Android app.

## How we built it

For our prototype, we used a knee brace to build the wearable with a bunch of hardware mounted on it. Specifically, we used an accelerometer (to measure thigh-calf angle), a low power micro-controller (Adafruit Flora), a Bluetooth Low Energy Module (low power Bluetooth) , a vibration motor (haptic feedback) and a Li-ion battery pack for the power. The hardware mounted on the knee-brace were either sewed or glued to ensure permanent fixture and repeatable measurements.
For the app implementation, we used a lot of online resources available on Android and other sites like Adafruit. The firmware on the micro-controller consisted of a comprehensive state machine to allow for integration with peripherals, data processing and communication.

## Challenges we ran into

One of the most difficult aspects in the project was getting Bluetooth communication to work between the Bluetooth Low Energy module and the Android smartphone. We spent nearly 5-6 hours to get it to work. We also faced calibration challenges early on with the knee-brace because the accelerometer is very sensitive and each time we retook measurements, the accelerometer position would change resulting in very different values. As a result, we glued the accelerometer to the knee brace ensure a permanent fixture. In addition, we wanted to create a user experience on the hardware and software side that would be simple yet descriptive enough to raise the patient's confidence and guide them towards a safe and productive physical therapy session.

## Accomplishments that we're proud of

Working as a team under pressure. It took us a while to keep going (understanding different sensors, micro-controllers, Android etc.) and there were a lot of moving parts in the solution that kept breaking down. Even though each member worked on their unique agenda item for this project, we were able to integrate various systems piece by piece without much trouble, or refactoring.
We knew that one of our biggest challenges would be to show that even in its first iteration smartPhy can achieve its foremost objective of **safe & productive physical therapy for our patients.** We are pleased with smartPhy's simple yet effective real time feedback to its user, where it doesn't give too much room for error while also guiding the patient towards a successful workout.
Most of all, we are happy that we stuck together during difficult times that ultimately helped us achieve the prototype we envisioned at the beginning of TreeHacks.

## What we learned

Hackathons are mostly about smart execution. Dividing up tasks and ensuring maximum parallelism will help greatly in overall project progress. It's okay to not have a perfect implementation- 36 hours is too less to build an amazing product. The idea is what matters.

## What's next for smartPhy : Making physical therapy affordable

We will take a break first! Beyond that, we are really interested in meeting up and interacting with people who are passionate about providing affordable physiotherapy. We think this space has huge potential for disruption and would love to work with equally passionate people.
## Inspiration

Physiotherapy is an important part of rehabilitation for physical injuries. A physiotherapist helps restore a patient’s mobility and prevents future injuries.

However, physiotherapy can be too expensive and time-consuming to repeatedly attend sessions at the clinic. Also, because of the pandemic, many Canadians were not able to regularly attend physiotherapy sessions because of limited service.

Enter Smart Heal! A low-cost, portable, and easy to use personal device that acts as your own virtual physiotherapist.

## What it does

Smart Heal gives you instructions and personally guides you through physiotherapy exercises in a fun and intuitive manner from the comfort of your home. The system consists of a Raspberry Pi and an IMU Sensor which is used to track the movement and orientation of any injured body part. Smart Heal also comes with a desktop app which I made to render a user’s current wrist orientation in a 3D virtual environment in real time.

The exercises are gamified to enhance the user experience. For example, I provide virtual targets that a user could try to hit and by doing that they end up moving their wrist to a proper exercise position. With your wrist, you can pretend you are controlling an airplane through different maneuvers that actually help exercise your wrist. All while being fun for all ages!

## How we built it

The Raspberry Pi is the central hub which is responsible for processing all sensor data. The Pi interfaces with the IMU and runs a Kalman Filter. The Kalman Filter, which I specifically tuned, makes sure that the raw IMU data is as smooth as can be. With the filter turned on, jerky hand movements are stabilized in the 3D visualizer.

The desktop visualizer app is built to display the IMU orientation feedback in the virtual environment. Once the desktop app is initialized, the app will automatically seek the Smart Heal device (Raspberry Pi) on the home wifi network and automatically connect with the IMU data stream. This communication backend is handled by ROS (Robot Operating System). Once the IMU data stream is received, the app will run through each required step of a specific wrist exercise (update text instructions and colours for the targets).
## Inspiration

Kingston's website is the place to go when having questions about having life in Kingston or when searching for events going on in the city, but navigating through the hundreds of the city's webpages for an answer can be gruesome. We were all interested in AI and wanted to challenge ourselves to build a chatbot website.

## What it does

Kingsley is a chatbot built to help residents in Kingston with their inquiries. It takes in user input, and responds with a helpful answer along with a link to where more information can be found on the city of Kingston website if applicable. It has an option for voice input and output for greater accessibility.

## How we built it

* Kingsley uses a GPT-3 model fine-tuned on data from the city of Kingston website.
* The data was scraped using Beautiful Soup.
* A GloVe model was used to find website links relevant to the user's question.
* Jaccard similarity was used to find relevant text that specifically mentioned key words in the user's question.
* Relevant texts were narrowed down and passed as part of the prompt to GPT-3 for an answer completion.
* The website along with the voice functionality were created using React.

## Challenges we ran into

The CityOfKingston website has a huge amount of pages, a lot of which are archived, calendars, or not very useful. OpenAI's API on the other hand only allowed a limited context. so to have the bot be able to read relevant pages as its context, we had to go through multiple methods of data filtering to find the relevant pages.
We spent a great amount of time implementing speech-to-text and text-to-speech for our webapp. Many of the solutions on the internet were of little help, and we tried using several npm packages before being successful in the end.

## Accomplishments that we're proud of

We successfully made a working chatbot!
And it references real facts! (sometimes)

## What we learned

Throughout the project, we gained experience working with various APIs. We learned how to use and combine different natural language processing techniques to optimize accuracy and computation time. We learned React hooks useState and useEffect, Javascript functions, and how to use React developer tools to debug components in Chrome. We figured out how to link backend Flask with frontend app, setup a domain, and use text-to-speech and speech-to-text libraries.

## What's next for Kingsley

Due to free trial limits, we chose to use the Ada GPT model for our chatbot. In the future if we had more credits, we could use a better version of GPT-3 in order to produce more relevant and helpful results.
We are also interested in expanding Kingsley to reference data from other websites. It can also be adapted as an extension or floating popup that can be used directly on top of Kingston's website.