## Inspiration

Choice is a mixed bag: having all the world's options at your hands means it's on *you* to spend the time needed to pick out the perfect choice from hundreds, especially if it's something as important as your literal degree. But who has the time today to be reading course descriptions, then career descriptions, then student course reviews (because the first two never help), then buried Reddit posts from five years ago, then wherever next that rabbit hole takes you?

Using the collective effort of thousands of students' feedback and thoughts, DirectU does the job for you. Based on real people's experiences, all the best courses for you—and specifically you—are laid out right in front of you.

## What it does

DirectU takes in your career aspirations and freely-written **practical** course preferences to parse through thousands of student course reviews, selecting the ones that match your profile and fitting them into a 4-year degree for you. Want to... take the ML courses that'll actually provide something of value? DirectU has your back. Minor with only the best possible courses for it? DirectU has your back. Pick the easiest way out? DirectU has! your! back!

After picking out all your preferences, the data is sent to our backend containing course reviews scraped from uwflow.com stored in a MongoDB database. Here, we make calls to Cohere's API to embed your liked courses and all courses available in the university, turning them into vectors. We compare the vector of each available course against the vectors of your liked courses using cosine similarity, generating a **similarity score** that represents how much the former matches the latter. Then, we take the courses that match your liked courses the most and suggest those to you in your dashboard!

❌ No more bullet-pointing every course name that sounds vaguely interesting on a separate Google Doc

❌ No more losing track of how many courses you can afford each term

❌ No more taking limiting personality quizzes with only four answer options to match you to a job

Once your courses have been laid out, you get access to an **aggregated summary of past students'** best tips, warnings, and advice for each course, generated from all past course reviews. You're told **why each course was recommended** with personalized pros and cons produced on the spot by Cohere based on your profile. If you don't like your primary recommendations, you're free to play around with our builder to test out secondary and tertiary recommendations!

## How we built it

* React (Vite) for the frontend dashboard
* Python and Flask for the backend
* Cohere APIs for text embedding, summarization, and generation
* MongoDB for our database
* Figma for workflow definition and wireframing
* Selenium for web scraping

## Challenges we ran into

One major challenge we ran into was problem definition. The premise of our original idea was way different than what was produced, being a more faculty-serving product. Further exploration into what our toolkit provided us, the delving of into workflows, and the ability to change tracks and re-use old work allowed us to build a product that would serve more people, more practically.

Another challenge we ran into was not taking into account the time taken for every API call. Originally, our backend made over 1,000 calls to Cohere's API to embed text every time a user initiated a session -- each API call took about one second and added to 17 minutes in total. We adapted our code and reduced the number of calls to Cohere's API down to just 2, single-handedly reduced the loading time for our application to 15 seconds, or a 99% decrease.

## Accomplishments that we're proud of

We're happy to be able to present a fully-functioning full-stack web application, from a student-friendly interface down to robust, optimized backend with stored data for future user sessions, especially considering half of our team is comprised of first-time hackers!

## What we learned

* Frontend: integration of 3rd party libraries for UI enhancement, DRY code principles, more effective state management
* Backend: implementation of Flask servers to connect to a React frontend, request error handling, web-scraping with Selenium, MongoDB for easy lookup of data, optimizing time complexity of complicated functions, efficient calls to APIs

## What's next for DirectU

We aim to expand the implementation of DirectU beyond Waterloo and UWFlow to other universities and their respective course evaluations. The service could potentially become a 3rd-party service that can be integrated into existing scheduling and timetabling products used by universities. We also want to enhance the prompt engineering to be able to produce course-specific insights in more parseable, concise chunks for better UX.
## What it does

Tabs allows you and your friends to easily keep track of your borrowed money, without needing to transfer any funds.

## Challenges we ran into

We originally wanted to do a hardware project, but didn't have all the supplies we needed so we struggled a little bit with coming up with a new idea.

## Accomplishments that we're proud of

We think the app looks really cute!

## What we learned

None of us have made an app before and this was our first time using React Native, so we all got to learn the basics of the language.
# see our presentation [here](https://docs.google.com/presentation/d/1AWFR0UEZ3NBi8W04uCgkNGMovDwHm_xRZ-3Zk3TC8-E/edit?usp=sharing)

## Inspiration

Without purchasing hardware, there are few ways to have contact-free interactions with your computer.
To make such technologies accessible to everyone, we created one of the first touch-less hardware-less means of computer control by employing machine learning and gesture analysis algorithms. Additionally, we wanted to make it as accessible as possible in order to reach a wide demographic of users and developers.

## What it does

Puppet uses machine learning technology such as k-means clustering in order to distinguish between different hand signs. Then, it interprets the hand-signs into computer inputs such as keys or mouse movements to allow the user to have full control without a physical keyboard or mouse.

## How we built it

Using OpenCV in order to capture the user's camera input and media-pipe to parse hand data, we could capture the relevant features of a user's hand. Once these features are extracted, they are fed into the k-means clustering algorithm (built with Sci-Kit Learn) to distinguish between different types of hand gestures. The hand gestures are then translated into specific computer commands which pair together AppleScript and PyAutoGUI to provide the user with the Puppet experience.

## Challenges we ran into

One major issue that we ran into was that in the first iteration of our k-means clustering algorithm the clusters were colliding. We fed into the model the distance of each on your hand from your wrist, and designed it to return the revenant gesture. Though we considered changing this to a coordinate-based system, we settled on changing the hand gestures to be more distinct with our current distance system. This was ultimately the best solution because it allowed us to keep a small model while increasing accuracy.

Mapping a finger position on camera to a point for the cursor on the screen was not as easy as expected. Because of inaccuracies in the hand detection among other things, the mouse was at first very shaky. Additionally, it was nearly impossible to reach the edges of the screen because your finger would not be detected near the edge of the camera's frame. In our Puppet implementation, we constantly *pursue* the desired cursor position instead of directly *tracking it* with the camera. Also, we scaled our coordinate system so it required less hand movement in order to reach the screen's edge.

## Accomplishments that we're proud of

We are proud of the gesture recognition model and motion algorithms we designed. We also take pride in the organization and execution of this project in such a short time.

## What we learned

A lot was discovered about the difficulties of utilizing hand gestures. From a data perspective, many of the gestures look very similar and it took us time to develop specific transformations, models and algorithms to parse our data into individual hand motions / signs.

Also, our team members possess diverse and separate skillsets in machine learning, mathematics and computer science. We can proudly say it required nearly all three of us to overcome any major issue presented. Because of this, we all leave here with a more advanced skillset in each of these areas and better continuity as a team.

## What's next for Puppet

Right now, Puppet can control presentations, the web, and your keyboard. In the future, puppet could control much more.

* Opportunities in education: Puppet provides a more interactive experience for controlling computers. This feature can be potentially utilized in elementary school classrooms to give kids hands-on learning with maps, science labs, and language.
* Opportunities in video games: As Puppet advances, it could provide game developers a way to create games wear the user interacts without a controller. Unlike technologies such as XBOX Kinect, it would require no additional hardware.
* Opportunities in virtual reality: Cheaper VR alternatives such as Google Cardboard could be paired with
Puppet to create a premium VR experience with at-home technology. This could be used in both examples described above.
* Opportunities in hospitals / public areas: People have been especially careful about avoiding germs lately. With Puppet, you won't need to touch any keyboard and mice shared by many doctors, providing a more sanitary way to use computers.