## Inspiration

We noticed that individual have difficulty achieving their fitness goals due to external factors such as work and sleep patterns. As a result, many fitness applications do not provide a realistic time frame for achieving said goal. We felt that having an application that customizes different fitness goals everyday to fit the individuals work schedule and sleep pattern from the night prior. This would be compared to their final goal and predict an estimate of when the user is able to achieve it.

## What it does

FullyFit uses both Fitbit and Muse API to get fitness and brain-waves (EEG data). Many features extracted from the data include steps taken, calories burnt, heart-rate, sleep and floors climbed each minute as well as each day. Also, data from Google calendar is extracted indicating the duration for which the user will be busy throughout the day. On the mental health side, FullyFit tracks brain wave activity via a Muse headband, allowing the app to adjust its exercise reminders based on mental stress and thinking.

## How we built it

We used machine learning to predict if the user is able to meet his/her step or calories burned goal. We scikit-learn machine learning library to test several algorithms like Decision Trees, Logistic Regression, Random Forest and Bagging Ensemble methods. Even though we got higher accuracy with both Random Forest and Bagging Ensemble methods, we used the Decision Trees for predicting if the user meets his daily fitness goals. Decision Trees ended up being really quick and also reasonably accurate for implementation.

## Challenges we ran into

There were some issues with the Fitbit api getting and handling intraday data for more than a month, so for demonstration purposes we analyzed only a months data.

The FullyFit is more accurate if the user has his work schedule posted in advance. This helps it better predict if the user would be able to meet their goal or not and notify them with suggestions.

## Things we learned

• We learned a lot about the different channels of signaling from the brain. Specifically: Delta waves which are most present during sleep.
Theta waves which are associated with sleep, very deep relaxation, and visualization. Alpha waves which occur when relaxed and calm. Beta waves which occur when, for example, actively thinking or problem-solving. Gamma waves which occur when involved in higher mental activity and consolidation of information.
• Our brainwave EEG API wrapper relied on old legacy Objective-C code that we had to interpret and make edits to.

• Interacting the Fitbit API

## Accomplishments that we're proud of

• Being able to implement both Fitbit and Muse API's in our application.
• Getting a high prediction accuracy with Bagging ensemble methods (96.5%), Random Forest (89%) and Decision Trees (84%)

## What's next for FullyFit

We would like to analyze much more data that we did now as well as optimizing our algorithms on many users. Currently using the user's calendar we only use the duration of busy hours during as a feature in training our machine learning models. We would like to build on this and make our system smarter by also taking in account the spread of the events. For example if the person is busy with work later in the day, it can compensate. We also like to expand upon the application of Muse as a mental health guide.
## Inspiration

We built an AI-powered physical trainer/therapist that provides real-time feedback and companionship as you exercise.

With the rise of digitization, people are spending more time indoors, leading to increasing trends of obesity and inactivity. We wanted to make it easier for people to get into health and fitness, ultimately improving lives by combating these the downsides of these trends. Our team built an AI-powered personal trainer that provides real-time feedback on exercise form using computer vision to analyze body movements. By leveraging state-of-the-art technologies, we aim to bring accessible and personalized fitness coaching to those who might feel isolated or have busy schedules, encouraging a more active lifestyle where it can otherwise be intimidating.

## What it does

Our AI personal trainer is a web application compatible with laptops equipped with webcams, designed to lower the barriers to fitness. When a user performs an exercise, the AI analyzes their movements in real-time using a pre-trained deep learning model. It provides immediate feedback in both textual and visual formats, correcting form and offering tips for improvement. The system tracks progress over time, offering personalized workout recommendations and gradually increasing difficulty based on performance. With voice guidance included, users receive tailored fitness coaching from anywhere, empowering them to stay consistent in their journey and helping to combat inactivity and lower the barriers of entry to the great world of fitness.

## How we built it

To create a solution that makes fitness more approachable, we focused on three main components:

Computer Vision Model: We utilized MediaPipe and its Pose Landmarks to detect and analyze users' body movements during exercises. MediaPipe's lightweight framework allowed us to efficiently assess posture and angles in real-time, which is crucial for providing immediate form correction and ensuring effective workouts.

Audio Interface: We initially planned to integrate OpenAI’s real-time API for seamless text-to-speech and speech-to-text capabilities, enhancing user interaction. However, due to time constraints with the newly released documentation, we implemented a hybrid solution using the Vosk API for speech recognition. While this approach introduced slightly higher latency, it enabled us to provide real-time auditory feedback, making the experience more engaging and accessible.

User Interface: The front end was built using React with JavaScript for a responsive and intuitive design. The backend, developed in Flask with Python, manages communication between the AI model, audio interface, and user data. This setup allows the machine learning models to run efficiently, providing smooth real-time feedback without the need for powerful hardware.

On the user interface side, the front end was built using React with JavaScript for a responsive and intuitive design. The backend, developed in Flask with Python, handles the communication between the AI model, audio interface, and the user's data.

## Challenges we ran into

One of the major challenges was integrating the real-time audio interface. We initially planned to use OpenAI’s real-time API, but due to the recent release of the documentation, we didn’t have enough time to fully implement it. This led us to use the Vosk API in conjunction with our system, which introduced increased codebase complexity in handling real-time feedback.

## Accomplishments that we're proud of

We're proud to have developed a functional AI personal trainer that combines computer vision and audio feedback to lower the barriers to fitness. Despite technical hurdles, we created a platform that can help people improve their health by making professional fitness guidance more accessible. Our application runs smoothly on various devices, making it easier for people to incorporate exercise into their daily lives and address the challenges of obesity and inactivity.

## What we learned

Through this project, we learned that sometimes you need to take a "back door" approach when the original plan doesn’t go as expected. Our experience with OpenAI’s real-time API taught us that even with exciting new technologies, there can be limitations or time constraints that require alternative solutions. In this case, we had to pivot to using the Vosk API alongside our real-time system, which, while not ideal, allowed us to continue forward. This experience reinforced the importance of flexibility and problem-solving when working on complex, innovative projects.

## What's next for AI Personal Trainer

Looking ahead, we plan to push the limits of the OpenAI real-time API to enhance performance and reduce latency, further improving the user experience. We aim to expand our exercise library and refine our feedback mechanisms to cater to users of all fitness levels. Developing a mobile app is also on our roadmap, increasing accessibility and convenience. Ultimately, we hope to collaborate with fitness professionals to validate and enhance our AI personal trainer, making it a reliable tool that encourages more people to lead healthier, active lives.
## Inspiration

The hardest part of working out is getting the correct technique. One of us worked with a personal trainer to perfect his barbell squat, and it took him *months*. While personal trainers are great, they're expensive and only work for certain people (who probably aren't college students). So, the idea of an "AI personal trainer" was born. What if an AI could 'watch' somebody do an exercise and provide feedback and tips on that person's technique, with just a few clicks? We're excited just thinking about the possibilities.

## What it does

The DeepSquat app allows the user to take a front-view and side-view video of their squat using voice commands ("start" and "stop"). A very deep neural network finds keypoints on the user's body during their squat, which are used to evaluate how well the user is performing on six different attributes— whether their knees are moving too forward, whether their back is rounding, etc.— and give them an overall "squat score". In addition to the score, we provide specific feedback to allow the user to correct their biggest mistakes and improve their squat form.

## How we built it

We extracted and modified a fully-convolutional neural network from the openpose library, and used python and Apple's CoreML library to convert the network to an iOS-readable format. We then wrote Swift code to parse the network's output into heatmaps for key points on the squatter's body (right-elbow, left-knee, etc.) Using the keypoints, we developed algorithms in Swift to evaluate the quality of a squat based on six attributes. This entire process was encapsulated by the app, which we developed in Swift. Frames from both the front-view and side-view video were inputted into the network and used for squat evaluation.

## Challenges we ran into

Understanding the output of the neural network was a significant challenge. Since openpose is designed to be used with a wrapper class, there was very little information on the raw output of the network (which we'd need to use it in iOS). Using the little documentation we had, exploring the data, and making inferences, we finally did figure it out.

In addition, since the network was so large, it was not feasible to predict at a fps rate. Using the voice commands, we could extract the frame at the middle of the video (the "bottom" of the squat) and extracted keypoints from this frame and the "start" frame to evaluate squat attributes, which ended up working well.

## Accomplishments that we're proud of

We're surprised but happy at how well the Swift algorithms we created for squat evaluation worked. In order to rate "how big" the mistake a user was making is, we had to do several squats ourselves and reproduce those mistakes on video. It was a significant time investment, but we got a good sense of what "tolerance" we could provide to mistakes (for example, how far can a user's knee go beyond their ankles in the image to be a moderate error vs. a serious one).

Testing the full app for the first time and seeing it rate our squats was also pretty awesome (one of us is a lot better than the other, but we won't say who).

## What we learned

We learned how to integrate deep learning into iOS apps, as well as the most common mistakes made while doing a squat. We also learned to effectively parallelize work and how to push through bugs, frustrating as they might be.

## What's next for DeepSquat

Expand to other exercises, suggest training routines, and more— who knows, it might be a full-on personal trainer someday.