Recommends exercises in real-time as the user is exercising based on their heart rate.
## Inspiration

We often do homework late when there are no TA hours scheduled, so we were thinking that having an easy way to reach out to tutors to get homework help would be very efficient and appreciated.

## What it does

It authenticates users with GAuth, asks them about whether they want to be a tutor or student, as well as relevant info. Then, it connects students who are waiting in queue to a tutor who can help so that they can discuss problems over a real-time chat.

## How we built it

We used Typescript, Firebase for authentication, Firestore for Database, React as our frontend library, Socket.io for real-time chat, and tentatively, peer.js for real-time video streams.

## Challenges we ran into

We tried to implement video streams with Peer.js in addition to chat, but we had difficulties with matching students and tutors through peer.js. Queuing and matching students with tutors was also challenging because there were mutiple client requests that had to be synced with a single data structure on the server.

## Accomplishments that we're proud of

The entire thing: authentication, how we learned a lot more about socket.io, and using webRTC for the live video stream.

## What we learned

We had some experience with React and web development before, but not a ton, so getting to build a functional help in less than 36 hours was very fulfilling and helped us accelerate our learning.

## What's next for HWHelp

We think that the idea is a neat concept, and we hope to fully integrate video streams, as well as a more complex and individualized matchmaking process.
## Inspiration

Kevin, one of our team members, is an enthusiastic basketball player, and frequently went to physiotherapy for a knee injury. He realized that a large part of the physiotherapy was actually away from the doctors' office - he needed to complete certain exercises with perfect form at home, in order to consistently improve his strength and balance. Through his story, we realized that so many people across North America require physiotherapy for far more severe conditions, be it from sports injuries, spinal chord injuries, or recovery from surgeries. Likewise, they will need to do at-home exercises individually, without supervision. For the patients, any repeated error can actually cause a deterioration in health. Therefore, we decided to leverage computer vision technology, to provide real-time feedback to patients to help them improve their rehab exercise form. At the same time, reports will be generated to the doctors, so that they may monitor the progress of patients and prioritize their urgency accordingly. We hope that phys.io will strengthen the feedback loop between patient and doctor, and accelerate the physical rehabilitation process for many North Americans.

## What it does

Through a mobile app, the patients will be able to film and upload a video of themselves completing a certain rehab exercise. The video then gets analyzed using a machine vision neural network, such that the movements of each body segment is measured. This raw data is then further processed to yield measurements and benchmarks for the relative success of the movement. In the app, the patients will receive a general score for their physical health as measured against their individual milestones, tips to improve the form, and a timeline of progress over the past weeks. At the same time, the same video analysis will be sent to the corresponding doctor's dashboard, in which the doctor will receive a more thorough medical analysis in how the patient's body is working together and a timeline of progress. The algorithm will also provide suggestions for the doctors' treatment of the patient, such as prioritizing a next appointment or increasing the difficulty of the exercise.

## How we built it

At the heart of the application is a Google Cloud Compute instance running together with a blobstore instance. The cloud compute cluster will ingest raw video posted to blobstore, and performs the machine vision analysis to yield the timescale body data.

We used Google App Engine and Firebase to create the rest of the web application and API's for the 2 types of clients we support: an iOS app, and a doctor's dashboard site. This manages day to day operations such as data lookup, and account management, but also provides the interface for the mobile application to send video data to the compute cluster. Furthermore, the app engine sinks processed results and feedback from blobstore and populates it into Firebase, which is used as the database and data-sync.

Finally, In order to generate reports for the doctors on the platform, we used stdlib's tasks and scale-able one-off functions to process results from Firebase over time and aggregate the data into complete chunks, which are then posted back into Firebase.

## Challenges we ran into

One of the major challenges we ran into was interfacing each technology with each other. Overall, the data pipeline involves many steps that, while each in itself is critical, also involve too many diverse platforms and technologies for the time we had to build it.

## What's next for phys.io

<https://docs.google.com/presentation/d/1Aq5esOgTQTXBWUPiorwaZxqXRFCekPFsfWqFSQvO3_c/edit?fbclid=IwAR0vqVDMYcX-e0-2MhiFKF400YdL8yelyKrLznvsMJVq_8HoEgjc-ePy8Hs#slide=id.g4838b09a0c_0_0>