**Inspiration**
The inspiration behind Block Touch comes from the desire to create an interactive and immersive experience for users by leveraging the power of Python computer vision. We aim to provide a unique and intuitive way for users to navigate and interact with a simulated world, using their hand movements to place blocks dynamically.

**What it does**
Block Touch utilizes Python computer vision to detect and interpret hand movements, allowing users to navigate within a simulated environment and place blocks in a virtual space. The application transforms real-world hand gestures into actions within the simulated world, offering a novel and engaging user experience.

**How we built it**
We built Block Touch by combining our expertise in Python programming and computer vision. The application uses computer vision algorithms to analyze and interpret the user's hand movements, translating them into commands that control the virtual world. We integrated libraries and frameworks to create a seamless and responsive interaction between the user and the simulated environment.

**Challenges we ran into**
While developing Block Touch, we encountered several challenges. Fine-tuning the computer vision algorithms to accurately recognize and interpret a variety of hand movements posed a significant challenge. Additionally, optimizing the application for real-time responsiveness and ensuring a smooth user experience posed technical hurdles that we had to overcome during the development process.

**Accomplishments that we're proud of**
We are proud to have successfully implemented a Python computer vision system that enables users to control and interact with a simulated world using their hand movements. Overcoming the challenges of accurately detecting and responding to various hand gestures represents a significant achievement for our team. The creation of an immersive and enjoyable user experience is a source of pride for us.

**What we learned**
During the development of Block Touch, we gained valuable insights into the complexities of integrating computer vision into interactive applications. We learned how to optimize algorithms for real-time performance, enhance gesture recognition accuracy, and create a seamless connection between the physical and virtual worlds.

**What's next for Block Touch**
In the future, we plan to expand the capabilities of Block Touch by incorporating more advanced features and functionalities. This includes refining the hand gesture recognition system, adding new interactions, and potentially integrating it with virtual reality (VR) environments. We aim to continue enhancing the user experience and exploring innovative ways to leverage computer vision for interactive applications.
## Inspiration

There are over 1.3 billion people in the world who live with some form of vision impairment. Often, retrieving small objects, especially off the ground, can be tedious for those without complete seeing ability. We wanted to create a solution for those people where technology can not only advise them, but physically guide their muscles in their daily life interacting with the world.

## What it does

ForeSight was meant to be as intuitive as possible in assisting people with their daily lives. This means tapping into people's sense of touch and guiding their muscles without the user having to think about it. ForeSight straps on the user's forearm and detects objects nearby. If the user begins to reach the object to grab it, ForeSight emits multidimensional vibrations in the armband which guide the muscles to move in the direction of the object to grab it without the user seeing its exact location.

## How we built it

This project involved multiple different disciplines and leveraged our entire team's past experience. We used a Logitech C615 camera and ran two different deep learning algorithms, specifically convolutional neural networks, to detect the object. One CNN was using the Tensorflow platform and served as our offline solution. Our other object detection algorithm uses AWS Sagemaker recorded significantly better results, but only works with an Internet connection. Thus, we use a two-sided approach where we used Tensorflow if no or weak connection was available and AWS Sagemaker if there was a suitable connection. The object detection and processing component can be done on any computer; specifically, a single-board computer like the NVIDIA Jetson Nano is a great choice. From there, we powered an ESP32 that drove the 14 different vibration motors that provided the haptic feedback in the armband. To supply power to the motors, we used transistor arrays to use power from an external Lithium-Ion battery. From a software side, we implemented an algorithm that accurately selected and set the right strength level of all the vibration motors. We used an approach that calculates the angular difference between the center of the object and the center of the frame as well as the distance between them to calculate the given vibration motors' strength. We also built a piece of simulation software that draws a circular histogram and graphs the usage of each vibration motor at any given time.

## Challenges we ran into

One of the major challenges we ran into was the capability of Deep Learning algorithms on the market. We had the impression that CNN could work like a “black box” and have nearly-perfect accuracy. However, this is not the case, and we experienced several glitches and inaccuracies. It then became our job to prevent these glitches from reaching the user’s experience.

Another challenge we ran into was fitting all of the hardware onto an armband without overwhelming the user. Especially on a body part as used as an arm, users prioritize movement and lack of weight on their devices. Therefore, we aimed to provide a device that is light and small.

## Accomplishments that we're proud of

We’re very proud that we were able to create a project that solves a true problem that a large population faces. In addition, we're proud that the project works and can't wait to take it further!

Specifically, we're particularly happy with the user experience of the project. The vibration motors work very well for influencing movement in the arms without involving too much thought or effort from the user.

## What we learned

We all learned how to implement a project that has mechanical, electrical, and software components and how to pack it seamlessly into one product.

From a more technical side, we gained more experience with Tensorflow and AWS. Also, working with various single board computers taught us a lot about how to use these in our projects.

## What's next for ForeSight

We’re looking forward to building our version 2 by ironing out some bugs and making the mechanical design more approachable. In addition, we’re looking at new features like facial recognition and voice control.
## Inspiration

The inspiration for this project stems from the growing need for more natural and intuitive ways for humans to interact with computers and digital environments. As technology becomes increasingly integrated into daily life, traditional input methods like keyboards and mice are sometimes insufficient for the nuanced interactions that modern applications demand. Gesture recognition offers a way to bridge the gap between humans and computers by allowing computers to understand and respond to natural human movements. By leveraging advanced computer vision techniques and machine learning models, such as those provided by the Google MediaPipe framework, developers can interpret hand gestures in real-time, making human-computer interaction more intuitive and engaging.

## What it does

The enhanced script combines gesture recognition via webcam using MediaPipe's hand tracking capabilities with real-time angle calculations between joints of the hand, and visualizes the results using PyGame. Additionally, it incorporates the ability to interact with external hardware through a serial connection, like an Arduino, for even more complex applications (which can be enhanced later on for future use).

## How we built it

Used OpenCV for image processing, MediaPipe for hand tracking, Pygame for visualization, and PySerial for Arduino communication.

## Challenges we ran into

The biggest challenge I ran into was integrating the real-time angle calculations into the PyGame simulation. Constructing a function to calculate the joint angles in an accurate, consistent manner required a bit of research into linear algebra that I did not expect to do beforehand.

## Accomplishments that we're proud of

I am proud that I was able to integrate multiple different libraries from OpenCV to Google Mediapipe to form a visualization tool that opens the door for new forms of human-computer interaction and aims to pave the way for a future where technology can seamlessly integrate into the fabric of human activity, enriching our interactions with the digital world.

## What we learned

I learned a lot more about how to apply CV libraries in order to enable gesture recognition and gained insight into how ML models such as KNN can leverage hand landmark data to better predict gestures as well.

## What's next for DigiGlove

I plan to continue improving the dynamic accuracy and responsiveness of DigiGlove. I hope to continue experimenting with how it may be used in tandem with arduino microcontrollers to collect real-time data and move away from its current static state.