## Inspiration

Who doesn't love studying with their friends; getting work done with countless hours of fun conversations. But when you have a looming midterm the next morning, maybe you shouldn't study with your friends?

That was until StudySync, a collaborative studying platform that optimizes your ability to study with your friends. By tracking attention and note taking using deep learning, users get assigned points for focusing with a leaderboard of everyone's score at the study session, allowing you to stay motivated to study and encourage your friends to not fall behind. With AI detection when users are going on their phone or about to sleep, StudySync uses AI voices to wake users up and yells at users to put their phone away.

StudySync understands the importance of active recall, and by inputting your notes/course text, StudySync will periodically give a quizzes based on content the group just finished studying. Our smart quiz system pairs you with peers to master difficult topics and lock in. With public and private study rooms for introverts and extroverts alike, our app creates a vibrant, supportive environment for learning—anywhere, anytime for a diverse group of people, thus, promoting diversity in educational attainment. However, StudySync knows the importance of breaks as well, and gives the group periodic breaks.

## What it does

StudySync isn't just another study app—it’s your new go-to for leveling up your learning game with a dose of fun. Imagine hoping into a collaborative study session with friends, where your attention is tracked by your camera using OpenCV and MediaPipe. Stay locked in and score points, but if you start scrolling through your phone, brace yourself—your points take a hit 😲. And for an extra twist, you'll get a dramatic cutscene with a voiceover (thanks, ElevenLabs!) calling you out, keeping things lighthearted but on track. The app can even tell when you're just plain exhausted, suggesting a break or nap when you need it most, with built-in Pomodoro timers for structured rest. When it's time to test your knowledge, StudySync serves up periodic quizzes based on the material you’ve uploaded, and the app toggles between quizzes and time intervals as you complete sections. Active recall keeps the pressure on, tossing in questions from earlier sections, while the app keeps track of your mistakes, pairing you with peers who can help fill in the gaps. It’s not just studying—it’s an interactive, engaging learning experience designed to keep you sharp and connected.

You're also able to create public/private study rooms – helpful for people who are introverted and want a study partner, and implement a system where you can input how many days you have before a test/quiz—it'll tailor your study plan to help you lock in on the topics that matter the most!

## How we built it

We built the backend with Node.js, Express.js, and used WebSockets for real-time features like room creation, joining rooms, sending messages and updating points. We also used HTTP (REST API) to handle regular POST requests with the Express framework. These requests return JSON data about rooms and their existence or provide a basic welcome message for users interacting with the backend directly, and is hosted on Vercel. We used other features to make the computer vision task work, such as OpenCV, and MediaPipe for facial detection and a DNN using TensorFlow, and for object detection we used a CNN dataset trained on 9,900 images. We also used ElevenLabs' Text-to-Speech API for a much more expressive voice.

The frontend is primarily built using JavaScript/TypeScript with React and Tailwind CSS. Socket.IO is used for real-time communication between the frontend and backend. For webcam access we used WebRTC, then we used a Fetch API for sending captured frames to the backend for attention analysis. We also created a leaderboard system to add in our gamification effect.

Not only that, but to keep track of user and session data, we implemented OAuth authentication using Clerk!

## Challenges we ran into

Firstly, we didn't have full team to begin with, as well as all our members having midterms during the time of the hackathon, in particular, one member in our team has her midterm on the second day of the hackathon, which was a difficult challenge as we navigate through how to make the best use of our time. Throughout this weekend, it has been essential to balance working on our project with our mental health, rest, and last-minute study. 2 of our members were also from Waterloo, so commuting back and forth was a difficult challenge for us. As we work in a team of three, we had to decide on a project idea, scope, features and get working on it immediately due to the limited time given the challenges above. Maximizing our productivity was difficult when some tasks depended on others.

We encountered challenges with merging Git commits, often overwriting each other's code and introducing bugs. To overcome this, we had to learn new technologies, techniques, and adapt quickly, and learned new technologies to make our ideas happen such as navigating around the computer vision aspect. We initially started with a variety of features we wanted to work with, and ran into a challenge where we had to decide which to pursue and how to pursue each individual features.

To navigate these challenges, we read documentations, used Stack Overflow, and spoke with sponsors and mentors. With some turnarounds, we solved them and executed out the features (mostly) according to plan!

## Accomplishments that we're proud of

We’re proud of the significant progress we’ve made in such a short time on a project we initially thought was overly ambitious. We've successfully reached the majority of our key milestones and have also picked up and learned different tools along the way, such as navigating around the computer vision aspect.

## What we learned

Tackling computer vision and integrating tools like OpenCV and MediaPipe introduced us to a host of new technical challenges. With that said, we learned to quickly adapt, troubleshoot, and leverage documentation and mentorship. Tackling CV and integrating different features with tools like ElevenLabs introduced us to a host of new technical challenges, but we learned to quickly adapt and troubleshoot.

## What's next for StudySync

StudySync is just getting started. Our next steps are focused on enhancing engagement and personalization. We’re working on new ways to customize study plans based on individual needs, such as tailoring quizzes and attention tracking based on how close exams are, indicated by users. We’ll also be expanding our social features, making it easier for introverted users to join public or private study rooms. Integrating active recall, cumulative quizzes, and collaborative teaching moments will ensure deeper learning and concept mastery. Plus, refining attention tracking is another major next step, with more advanced metrics to help users manage fatigue and focus. We hope to incorporate more features like uploadable materials, interactive cut scenes, and real-time study performance insights. Stay tuned for the next wave of immersive, collaborative, effective study experiences!
## Inspiration

The idea for Study Buddy was born out of a personal struggle that many students (including ourselves),professionals, and lifelong learners can relate to—how to make studying less of a chore and more of an engaging experience. Despite the fact that we love learning, we often find ourselves losing motivation with the monotonous routine of flipping through pages and highlighting notes. That's where the idea originated to make learning a fun and dynamic experience like listening to a song or playing a game

An app that could turn the tedious process of reviewing notes into something exciting - where a quick scan of a book could turn facts into trivia games, where a dry chapter of history could become a short podcast, and complex science terms could become a catchy song.

Study Buddy is a personalized AI study assistant for students, professionals, or anyone looking to turn study sessions into engaging, memorable experiences. It bridges the gap between traditional learning and modern technology, making education both fun and effective.

## What it does

Study Buddy is an innovative educational app designed to revolutionize the way people learn by transforming traditional study materials into engaging, personalized experiences. The app uses real-time data from pictures or scans of textbooks, notes, or any reading material and converts them into interactive games, bite-sized podcasts, or even custom songs.

The process is simple: users scan or upload content, and Study Buddy analyzes the information to create tailored learning experiences. Whether it's turning a chapter of history into a trivia game, summarizing a biology lesson into a short podcast, or converting complex formulas into catchy songs, Study Buddy offers a unique and fun way to absorb knowledge.

## How we built it

The Study Buddy app was built using an advanced and dynamic tech stack designed to deliver a highly personalized and engaging learning experience.

* It leverages Gemini to generate content for both game creation and song lyrics based on user-provided material, ensuring real-time adaptation to different learning topics.

-To make the experience even more immersive, HumeAI adds human-like voices to sing the generated lyrics.

-Deepgram enables word-level audio editing, ensuring perfect pitch and fluidity in the songs.

* For game mechanics, Fetch AI introduces smart agents with unique personas that adjust difficulty as users level up, creating an adaptive learning curve.

-On the backend, the app utilizes a RAG-powered retrieval system to handle input data efficiently, offering fast and accurate content generation.

-This is powered by SingleStore Database, ensuring rapid processing and seamless user interaction, making Study Buddy a powerful, interactive learning tool.

## Challenges we ran into

The major challenge lies in creating highly personalized learning experiences. The app must accurately interpret user input, whether it's a scanned text or an image, and then convert it into engaging formats like games, podcasts, or songs. This requires robust natural language processing (NLP) capabilities, and ensuring the accuracy and relevancy of the content generated by Gemini or RAG-powered retrieval systems can be difficult, particularly with diverse and complex study materials.

Maintaining pitch-perfect songs with Deepgram and ensuring the game difficulty adjusts naturally with Fetch AI's smart agents adds further layers of complexity in fine-tuning.

## Accomplishments that we're proud of

Building Study Buddy comes with a range of accomplishments that anyone involved in its development would be incredibly proud of. First and foremost is the creation of a cutting-edge tool that fundamentally reimagines how people learn by merging education with entertainment in an innovative way. Successfully developing algorithms that can analyze real-time data from text, images, or scanned content and convert them into interactive, user-friendly formats—such as games, podcasts, or songs—is a technical achievement in itself. Additionally, the app's ability to cater to different learning styles and preferences represents a major win for personalized education. The team would also take pride in the user impact, knowing that Study Buddy makes learning accessible, fun, and effective for students of all ages. Ultimately, the most fulfilling accomplishment would be empowering users to transform their studying experience from something tedious into an engaging journey, helping them retain knowledge in ways that suit them best.

## What we learned

Building the Study Buddy app is a transformative learning experience, both from a technical and a personal perspective. On the technical side, we gained a deeper understanding of how to integrate complex machine learning algorithms to process real-time data, such as scanning text or images, and dynamically converting it into interactive formats like games, podcasts, or songs. The challenge of designing a platform that can adapt to diverse learning styles—visual and auditory teaches valuable lessons in user experience (UX) design, requiring a deep understanding of how different people interact with educational tools.

From a project management standpoint, working on Study Buddy involves navigating the collaboration between multiple disciplines, such as education, entertainment, and technology, which fosters teamwork and interdisciplinary communication skills. We learn how to balance innovation with practicality—ensuring the app remains both cutting-edge and user-friendly. Moreover, building an app that aims to improve education requires constant empathy and feedback from users, teaching the importance of user-centric design and iteration.

## What's next for Study Buddy

The future plan for Study Buddy focuses on expanding its capabilities and improving user experience.

* Augmented Reality (AR) Integration: Bring a new dimension to learning by incorporating AR. This could allow users to scan physical textbooks and get interactive 3D models or visualizations of complex concepts, making learning more immersive.
* Collaborative Learning Features: Add multiplayer or peer-learning modes where users can collaborate or compete with friends in games, trivia challenges, or joint quizzes. This can introduce a social element to the learning experience, enhancing engagement.
* Advanced Voice Customization: Further improve the HumeAI and Deepgram integration, allowing users to customize the voice singing their notes with more control over accents, tones, and expressions, or even choose from a broader range of celebrity voices or favorite artists.
* Leaderboards and Competitions: Introduce global or regional leaderboards to foster a sense of competition among users, with rewards or recognition for top learners in different subjects or topics.
## Inspiration

While we were doing preliminary research, we had found overwhelming amounts of evidence of mental health deterioration as a consequence of life-altering lockdown restrictions. Academic research has shown that adolescents depend on friendship to maintain a sense of self-worth and to manage anxiety and depression. Intimate exchanges and self-esteem support significantly increased long-term self worth and decreased depression.

While people do have virtual classes and social media, some still had trouble feeling close with anyone. This is because conventional forums and social media did not provide a safe space for conversation beyond the superficial. User research also revealed that friendships formed by physical proximity don't necessarily make people feel understood and resulted in feelings of loneliness anyway. Proximity friendships formed in virtual classes also felt shallow in the sense that it only lasted for the duration of the online class.

With this in mind, we wanted to create a platform that encouraged users to talk about their true feelings, and maximize the chance that the user would get heartfelt and intimate replies.

## What it does

Reach is an anonymous forum that is focused on providing a safe space for people to talk about their personal struggles. The anonymity encourages people to speak from the heart. Users can talk about their struggles and categorize them, making it easy for others in similar positions to find these posts and build a sense of closeness with the poster. People with similar struggles have a higher chance of truly understanding each other. Since ill-mannered users can exploit anonymity, there is a tone analyzer that will block posts and replies that contain mean-spirited content from being published while still letting posts of a venting nature through. There is also ReCAPTCHA to block bot spamming.

## How we built it

* Wireframing and Prototyping: Figma
* Backend: Java 11 with Spring Boot
* Database: PostgresSQL
* Frontend: Bootstrap
* External Integration: Recaptcha v3 and IBM Watson - Tone Analyzer
* Cloud: Heroku

## Challenges we ran into

We initially found it a bit difficult to come up with ideas for a solution to the problem of helping people communicate. A plan for a VR space for 'physical' chatting was also scrapped due to time constraints, as we didn't have enough time left to do it by the time we came up with the idea. We knew that forums were already common enough on the internet, so it took time to come up with a product strategy that differentiated us. (Also, time zone issues. The UXer is Australian. They took caffeine pills and still fell asleep.)

## Accomplishments that we're proud of

Finishing it on time, for starters. It felt like we had a bit of a scope problem at the start when deciding to make a functional forum with all these extra features, but I think we pulled it off. The UXer also iterated about 30 screens in total. The Figma file is *messy.*

## What we learned

As our first virtual hackathon, this has been a learning experience for remote collaborative work.
UXer: I feel like i've gotten better at speedrunning the UX process even quicker than before. It usually takes a while for me to get started on things. I'm also not quite familiar with code (I only know python), so watching the dev work and finding out what kind of things people can code was exciting to see.

# What's next for Reach

If this was a real project, we'd work on implementing VR features for those who missed certain physical spaces. We'd also try to work out improvements to moderation, and perhaps a voice chat for users who want to call.