## Inspiration

A team member saw their mom working and repeating the same steps over and over on her computer. Open an excel sheet, copy 10 lines, tab into another sheet, paste, scroll, repeat. These monotonous tasks are a time waster which is terrible especially since the technology exists to automate these tasks. The problem is that most people aren't aware, have forgotten, or find it to complicated which is why we aim to reintroduce this technology in a simple manner that anyone can use. An application that could automate such repetitive tasks would not only save time but also could have other uses. For example, individuals that have arthritis may find it challenging to complete tasks on the computer that require repetitive movement on an everyday basis. We offer an application that can take these repetitive movements and simplify it to 1 click.

## What it does

Our program, dubbed "Ekko", serves to:

1. Reduce menial/repetitive tasks
2. Increase accessibility to mouse and keyboard actions for those with diseases/disabilities
3. Aids those who aren’t technologically literate
Users will be able to record any series of clicks, cursor movements and key presses as they would like, and save this sequence. They would then be able to play it back at will whenever they desire.

## How we built it

To create this application we used various Python libraries. In the backend we had to implement the *Pynput* library which allowed tracking of clicks, cursor movement and key presses. We have individual threads set up for monitoring both keyboard and mouse actions at the same time. In the front end, we used QtDesigner and *PyQt5* to design our GUI. Using Python
Libraries/Software: *Pynput*, *PyQt5*, QtDesigner, *Multithreading*
Used *Pynput* to track and record cursor movement and keypresses and clicks
Used *PyQt5* and *QtDesigner* to design the GUI

## Challenges we ran into

Initially, without the use of the *Multithreading* library our program would crash after one use. It took a lot of time to figure out what the exact problem was and how to allow for recording of cursor and keyboard inputs while the Python program was running. Getting the GUI to interface with the recording/playing scripts was also annoying. A bit of a funny obstacle we encountered was that while 1 team member was following a tutorial he didn't realize they were alternating between Spanish and English while writing code which led to a few confusing errors.

## Accomplishments that we're proud of

We had created a timeline for our project with deadlines for specific tasks. We were able to stick to this schedule and complete our project in a timely manner, even though we were busy. We also successfully implemented a program with a working GUI. Finally, it was the first hackathon experience for 2 of our members so it was nice to have a finished product which often doesn't happen on an individual's first hackathon.

## What we learned

We learned many new tech skills such as how to create GUI's, using threads, how to track keyboard and mouse input. We also learned important soft skills such as time management. These skills will be useful in whatever future projects we decide to pursue.

## What's next for Ekko: Automate Your Computer

Implementing voice recognition to make automating tasks more accessible. The movements of the cursor can also be more accurate during repetition.
## Inspiration

With the sudden move to online videoconferencing, presenters and audiences have been faced with a number of challenges. Foremost among these is a lack of engagement between presenters and the audience, which is exacerbated by a lack of gestures and body language. As first year students, we have seen this negatively impact our learning throughout both high school and our first year of University. In fact, many studies, such as [link](https://dl.acm.org/doi/abs/10.1145/2647868.2654909), emphasize the direct link between gestures and audience engagement. As such, we wanted to find a way to give presenters the opportunity to increase audience engagement through bringing natural presentations techniques to videoconferencing.

## What it does

PGTCV is a Python program that allows users to move back from their camera and incorporate body language into their presentations without losing fundamental control. In its current state, the Python script uses camera information to determine whether a user needs their slides to be moved forwards or backwards. To trigger these actions, users raise their left fist to enable the program to listen for instructions. They can then swipe with their palm out to the left or to the right to trigger a forwards or backwards slide change. This process allows users to use common body language and hand gestures without accidentally triggering the controls.

## How we built it

After fetching webcam data through OpenCV2, we use Google's MediaPipe library to receive a co-ordinate representation of any hands on-screen. This is then fed through a pre-trained algorithm to listen for any left-hand controlling gestures. Once a control gesture is found, we track right-hand motion gestures, and simulate the relevant keyboard input using pynput in whatever application the user is focused on. The application also creates a new virtual camera in a host Windows machine using pyvirtualcam and Unity Capture since Windows only allows one application to use any single camera device. The virtual camera can be used by any videoconferencing application.

## Challenges we ran into

Inability to get IDEs working. Mac M1 chip not supporting Tensorflow. Inability to use webcam in multiple applications at once. Setting up right-hand gesture recognition with realistic thresholds.

## Accomplishments that we're proud of

Successfully implementing our idea in our first hackathon. Getting a functional and relatively bug-free version of the program running with time to spare. Learning to successfully work with a number of technologies that we previously had no experience with (everything other than Python).

## What we learned

A number of relevant technologies. Implementing simple computer vision algorithms. Taking code from idea to functional prototype in a limited amount of time.

## What's next for Presentation Gestures Through Computer Vision (PGTCV)

A better name. Implementation of a wider range of gestures. Optimization of algorithms. Increased accuracy in detecting gestures. Implementation into existing videoconferencing applications.
## Inspiration

**Handwriting is such a beautiful form of art that is unique to every person, yet unfortunately, it is not accessible to everyone.**

[Parkinson’s](www.parkinson.org/Understanding-Parkinsons/Statistics) affects nearly 1 million people in the United States and more than 6 million people worldwide. For people who struggle with fine motor skills, picking up a pencil and writing is easier said than done. *We want to change that.*

We were inspired to help people who find difficulty in writing, whether it be those with Parkinson's or anyone else who has lost the ability to write with ease. We believe anyone, whether it be those suffering terminal illnesses, amputated limbs, or simply anyone who cannot write easily, should all be able to experience the joy of writing!

## What it does

Hand Spoken is an innovative solution that combines the ease of writing with the beauty of an individual's unique handwriting.

All you need to use our desktop application is an old handwritten letter saved by you! Simply pick up your paper of handwriting (or handwriting of choice) and take a picture. After submitting the picture into our website database, you are all set. Then, simply speak into the computer either using a microphone or a voice technology device. The user of the desktop application will automatically see their text appear on the screen in their own personal handwriting font! They can then save their message for later use.

## How we built it

We created a desktop application using C# with Visual Studio's WinForm framework. Handwriting images uploaded to the application is sent via HTTP request to the backend, where a python server identifies each letter using pytesseract. The recognized letters are used to generate a custom font, which is saved to the server. Future audio files recorded by the frontend are also sent into the backend, at which point AWS Transcribe services are contacted, giving us the transcribed text. This text is then processed using the custom handwriting font, being eventually returned to the frontend, ready to be downloaded by the user.

## Challenges we ran into

One main challenge our team ran into was working with pytesseract. To overcome this obstacle, we made sure we worked collaboratively as a team to divide roles and learn how to use these exciting softwares.

## Accomplishments that we're proud of

We are proud of creating a usable and functional database that incorporates UX/UI design!

## What we learned

Not only did we learn lots about OCR (Optical Character Recognition) and AWS Transcribe services, but we learned how to collaborate effectively as a team and maximize each other's strengths.

## What's next for Hand Spoken

Building upon on our idea and creating accessibility **for all** through the use of technology!