## About the Project

We are a bunch of amateur players who love playing chess, but over time we noticed that our improvement has become stagnant. Like many college students, we neither have the time nor the financial means to invest in professional coaching to take our game to the next level. This frustration sparked the idea behind **Pawn Up**—a project built to help players like us break through the plateau and improve their chess skills in their own time, without expensive coaches or overwhelming resources.

### What Inspired Us

As passionate chess players, we struggled with finding affordable and effective ways to improve. Chess can be an expensive hobby if you want to seek professional help or guidance. The available tools often lacked the depth we needed or came with hefty price tags. We wanted something that would provide personalized feedback, targeted training, and insights that anyone could access—regardless of their financial situation.

### How We Built It

We started by integrating **Lichess authentication** to fetch a user's game history, allowing them to directly analyze their own performance. With **Groq** and **Llama3.1**, we leveraged AI to categorize mistakes, generate feedback, and suggest relevant puzzles to help users train and improve. We also levergae **ChromaDB** for vector search features and **Gemini pro** and **Gemini embedding**

Our project features four key components:

* **Analyze**: Fetches the user's last 10 games, provides analysis on each move, and visualizes a heatmap showing the performance of legal moves for each piece. Users can also interact with the game for deeper analysis.
* **Train**: Using AI, the system analyzes the user's past games and suggests categorized puzzles that target areas of improvement.
* **Search**: We created a vector database storing thousands of grandmaster games. Users can search for specific games and replay them with detailed analysis, just like with their own games.
* **Upload**: Users can upload their own chess games and perform the same analyses and training as with the **Search** feature.

### What We Learned

Throughout the development of **Pawn Up**, we gained a deeper understanding of AI-powered analysis and how to work with complex game datasets. We learned how to integrate chess engines, handle large amounts of data, and create user-friendly interfaces. Additionally, we explored how LLMs (large language models) can provide meaningful feedback and how vector databases can be used to store and retrieve massive datasets efficiently.

### Challenges We Faced

One of the main challenges we encountered was making the AI feedback meaningful for players across various skill levels. It was crucial that the system didn’t just provide generic advice but rather tailored suggestions that were both practical and actionable. Handling large amounts of chess data efficiently, without compromising on speed and usability, also posed a challenge. Building the vector database to store and search through grandmaster games was a particularly challenging but rewarding experience.

Despite these hurdles, we’re proud of what we’ve built. **Pawn Up** is the solution we wish we had when we first started hitting that plateau in our chess journeys, and we hope it can help others as well.
>
> Domain.com entry: bearable9to5.online
>
>
>

## Inspiration

As our team members were getting to know each other, one of us commented on how tiring meeting on zoom is and how we wished we could be hacking together in person. Another teammate mentioned the concept of “Zoom Fatigue” and how it is a real topic studied by psychologists. As the discussion elevated, we wondered if there was anything we could do to help combat this issue, and conducted lots of research. We discovered that Zoom fatigue stems from how our brains process information over video.

>
> “Consider how long we stare at the Zoom screen or camera during a 15-minute standup. Most of us dare not turn away—even for a moment—because we worry our colleagues will think we’re distracted. The lack of visual breaks in virtual meetings strains both our eyes and brains, making it harder to stay focused.” (HBR.org)
>
>
>

We all related with this fact and agreed that Zoom and other video calling software should implement a “take a break” feature for both physical and mental health purposes. We also wondered what other features would help to ease the pressures of the video calling experience and make it more engaging. We were eager to uncover the ways in which we could improve one of the most dreaded but necessary tasks of today's virtual world.

---

## What it does

Bearable 9 to 5 is a work from home companion that increases the wellbeing of those who have “Zoom Fatigue” caused by the overload of video call meetings. This software wrapper utilizes a virtual camera that can run on any video calling program such as Zoom and Teams. It combats Zoom fatigue by reminding and enabling users to take solo or group breaks during meetings to stretch as well as gives them tips to improve their wellbeing. Some of these tips include tracking their posture and notifying them when they are slouching, as well as using machine learning models designed to detect human emotion to uplift the user with positivity if it detects they are glum or upset.

---

## How we built it

We chose Python as our programming language of choice as Python is powerful, beginner friendly, and has a wide selection of library support. Using pyvirtualcam, we were able to integrate with Zoom and connect it with a virtual camera which we could capture and process every frame. We used multithreading in combination with opencv to process video frame data, and allowed asynchronous control of the virtual camera while providing an user interface at the same time. We have also connected to Google Cloud Processing for additional API support, specifically Google Cloud Vision and Emotions API, to periodically check the stress level of the user during their video meeting, and on detection of negative emotions such as anger or grief, a pop-up would be displayed for a smile reminder.

---

## Challenges we ran into

One of the challenges we ran into was getting the pyvirtualcam library to work on all of our team members computers. After spending hours troubleshooting, we were able to get it working for 2 out of the 4 people on our team and implemented a paired programming approach. Another challenge was the fact that only one person on our team had used image capturing libraries before, so it was a learning curve to overcome. A specific technical challenge we had to overcome was disabling the user camera and displaying a “User is taking a break” at the same time as the user interacting with the gui. We were able to accomplish this by using threads in python to allow both processes to occur at once.

---

## Accomplishments that we're proud of

Considering that 3 out of 4 team members didn’t have experience with using image capturing libraries and Python itself required some relearning, this project as a whole was a great success. Our team was diverse in that we had different genders and different experience levels as well as a first time hacker, and it was really interesting and fun to be able to work with and support each other with everyone doing a sizeable chunk of work. Ultimately, we are very proud to have a complete product on github by the end of this hackathon that works very well. This team worked very hard to accomplish its goals and we’re proud to have implemented many relevant features to help combat “Zoom Fatigue” such as emotion detection and posture detection which were both powered by Google Cloud Processing. We’re also proud to have been able to get the “Share Stretching” feature to work with pre-recorded stretching and exercise videos playing directly within the video call, this was no simple task.

---

## What we learned

Initially, our team was leaning more towards a web app as that is where our expertise lay. However, we decided that we wanted to tackle this project even though we knew it would be harder and it has taught us more in one weekend than we could have ever imagined. Some new concepts we learned this weekend include:

* Using opencv for image and video processing
* Using python to run a virtual camera
* Remote collaboration and paired programming techniques
* Utilizing threads to run multiple processes at once
* Google cloud Processing (emotions, posture detection)

A general concept we learned was how successful we can be if we stay determined and focused on the project, and work hard throughout. We are very proud and fulfilled that we chose a harder project to tackle for our team because the learning outcome has been incredible.

---

## What's next for bearable 9to5

We definitely want to continue working on bearable 9to5 after this hackathon. One feature we want to implement is including an avatar that can speak on your behalf when you don’t want to be on camera and moves based on your facial recognition. This is helpful because it shows attendees that you are engaged in the meeting, but you don’t necessarily have to show your camera which can cause stress.
## Inspiration

[Cursorless](https://www.cursorless.org/) is a voice interface for manipulating text (e.g., code). We saw its potential as a bold new interface for text editing. However, it is very unintuitive and learning Cursorless amounts to learning a new language, with unnatural and complicated syntax.
This was the inspiration behind Verbalist. We want to harness the power of voice (and AI) to greatly improve productivity while editing text, especially code.
Most other AI products access user data. We also want to ensure data security of our product.

## What it does

Verbalist is a VSCode extension that enables the use of voice to edit their code. After a user downloads and configures the extension, users can record small voice snippets describing the high-level actions they want to take on text. Then, our AI models decide the specific actions to execute in order to do the high-levels actions--all without processing the content of the file.

## Challenges and what we learned

We learned some limitations of using large-language models on difficult, real-world tasks. For example, the LLM model we used often struggled to identify a correct, intuitive sequence of actions to perform the user's specified action. We spent a long time refining prompts; we learned that our final results were very sensitive to the quality of our prompts.
We also spent a while setting up the interaction between our main extension TypeScript file and our Python file, which handled the recording and AI processing. Through this process, we learned how to set up inter-process communication and extensively using the standard libraries (e.g., input/output streams) of both Python and TypeScript.

## Accomplishments we're proud of

Our extension allows users to use natural language to manipulate collections of lines and perform simple find-and-replace operations. T
We also built on top of the VSCode text editing API to allow for higher-level operations without providing any file contents to AI.

## What's next

The concepts behind this prototype can easily be extended to a fully-functional extension that adds a functionality not present in any other software today. We can implement more high-level, detailed actions for the AI to perform; for example, the ability to rename a variable, surround an expression in parentheses, or perform actions across multiple files. The voice interface can become a natural extension of the keyboard, one that allows programmers to spend less time thinking and more time doing.