## Inspiration

1.3 billion People have some sort of vision impairment. They face difficulties in simple day to day task like reading, recognizing faces, objects, etc. Despite the huge number surprisingly there are only a few devices in the market to aid them, which can be hard on the pocket (5000$ - 10000$!!). These devices essentially just magnify the images and only help those with mild to moderate impairment. There is no product in circulation for those who are completely blind.

## What it does

The Third Eye brings a plethora of features at just 5% the cost. We set our minds to come up with a device that provides much more than just a sense of sight and most importantly is affordable to all. We see this product as an edge cutting technology for futuristic development of Assistive technologies.

## Feature List

**Guidance** - ***Uses haptic feedback to navigate the user to the room they choose avoiding all obstacles***. In fact, it's a soothing robotic head massage guiding you through the obstacles around you. Believe me, you're going to love it.

**Home Automation** - ***Provides full control over all house appliances***. With our device, you can just call out **Alexa** or even using the mobile app and tell it to switch off those appliances directly from the bed now.

**Face Recognition** - Recognize friends (even their emotions (P.S: Thanks to **Cloud Vision's** accurate facial recognitions!). Found someone new? Don't worry, on your command, we register their face in our database to ensure the next meeting is no more anonymous and awkward!

**Event Description** - ***Describes the activity taking place***. A group of people waving somewhere and you still not sure what's going on next to you? Fear not, we have made this device very much alive as this specific feature gives speech feedback describing the scenic beauty around you. [Thanks to **Microsoft Azure API**]

**Read Up** - You don't need to spend some extra bucks for blind based products like braille devices. Whether it be general printed text or a handwritten note. With the help of **Google Cloud Vision**, we got you covered from both ends. **Read up** not only decodes the text from the image but using **Google text to speech**, we also convert the decoded data into a speech so that the blind person won't face any difficulty reading any kind of books or notes they want.

**Object Locator** - Okay, so whether we are blind or not, we all have this bad habit of misplacing things. Even with the two eyes, sometimes it's too much pain to find the misplaced things in our rooms. And so, we have added the feature of locating most generic objects within the camera frame with its approximate location. You can either ask for a specific object which you're looking for or just get the feedback of all the objects **Google Cloud Vision** has found for you.

**Text-a-Friend** - In the world full of virtuality and social media, we can be pushed back if we don't have access to the fully connected online world. Typing could be difficult at times if you have vision issues and so using **Twilio API** now you can easily send text messages to saved contacts.

**SOS** - Okay, so I am in an emergency, but I can't find and trigger the SOS feature!? Again, thanks to the **Twilio** messaging and phone call services, with the help of our image and sensor data, now any blind person can ***Quickly intimate the authorities of the emergency along with their GPS location***. (This includes auto-detection of hazards too)

**EZ Shoppe** - It's not an easy job for a blind person to access ATMs or perform monetary transactions independently. And so, taking this into consideration, with the help of superbly designed **Capital One Hackathon API**, we have created a **server-based blockchain** transaction system which adds ease to your shopping without being worried about anything. Currently, the server integrated module supports **customer addition, account addition, person to person transactions, merchant transactions, balance check and info, withdrawals and secure payment to vendors**. No need of worrying about individual items, just with one QR scan, your entire shopping list is generated along with the vendor information and the total billing amount.

**What's up Doc** - Monitoring heart pulse rate and using online datasets, we devised a machine learning algorithm and classified labels which tells about the person's health. These labels include: "Athletic", "Excellent", 'Good", "Above Average", "Average", "Below Average" and "Poor". The function takes age, heart rate, and gender as an argument and performs the computation to provide you with the best current condition of your heart pulse rate.

\*All features above can be triggered from Phone via voice, Alexa echo dot and even the wearable itself.
\*\*Output information is relayed via headphones and Alexa.

## How we built it

Retrofit Devices (NodeMCU) fit behind switchboards and allow them to be controlled remotely.

The **RSSI guidance uses Wi-Fi signal intensity** to triangulate its position. Ultrasonic sensor and camera detects obstacles (**OpenCV**) and runs Left and Right haptic motors according to closeness to device and position of the obstacle.

We used **dlib computer vision library** to record and extract features to perform **facial recognition**.

**Microsoft Azure Cloud services** takes a series of images to describe the activity taking place.

We used **Optical Character Recognition (Google Cloud)** for Text To Speech Output.

We used **Google Cloud Vision** which classifies and locates the object.

**Twilio API** sends the alert using GPS from Phone when a hazard is detected by the **Google Cloud Vision API**.

QR Scanner scans the QR Code and uses **Capital One API** to make secure and fast transactions in a **BlockChain Network**.

Pulse Sensor data is taken and sent to the server where it is analysed using ML models from **AWS SageMaker** to make the health predictions.

## Challenges we ran into

Making individual modules was a bit easier but integrating them all together into one hardware (Raspberry Pi) and getting them to work was something really challenging to us.

## Accomplishments that we're proud of

The number of features we successfully integrated to prototype level.

## What we learned

We learned to trust in ourselves and our teammates and that when we do that there's nothing we can't accomplish.

## What's next for The Third Eye

Adding a personal assistant to up the game and so much more.
Every person has potential they deserve to unleash; we pledge to level the playfield by taking this initiative forward and strongly urge you to help us in this undertaking.
## Inspiration

Our team focuses primarily on implementing innovative technologies to spark positive change. In line with these ideals, we decided to explore the potential impact of a text reading software that not only provided an additional layer of accessibility for blind and deafblind individuals. Through these efforts, we discovered that despite previous attempts to solve similar problems, solutions were often extremely expensive and incongruous. According to many visually challenged advocates these technologies often lacked real-world application and were specific to online texts or readings which limited their opportunities in everyday tasks like reading books or scanning over menus. Upon further research into afflicted groups, we discovered that there was additionally a large population of people who were both deaf and blind which stopped them from utilizing any forms of auditory input as an alternative, significantly obstructing their means of communication. Employing a very human centered design rooted in various personal accounts and professional testimony, we were able to develop a universal design that provides the visually and dual sensory impaired to experience the world from a new lens. By creating a handheld text to braille and speech generator, we are able to revolutionize the prospects of interpersonal communication for these individuals.

## What it does

This solution utilizes a singular piece with two modules, a video camera to decipher text, and a set of solenoids that imitates a standard Grade 2 Braille Grid. This portable accessory is intended to be utilized by a visually impaired or blind deaf individual when they’re attempting to analyze a physical text. This three finger supplement, equipped with a live action camera and sensitive solenoid components, is capable of utilizing a live camera feed to discern the diction of a physical text. Moving forward, the scanned text is moved to an A.I application to clean up the text for either auditory or sensory output in the form of TTS and braille. The text is then adapted into an audio format through a web application or to the classic 6 cells present in the Braille dictionary. Users are given a brief moment to make sense of each braille letter before the system automatically iterates through the remainder of the text. This technology effectively provides users with an alternative method to receive information that isn’t ordinarily accessible to them, granting a more authentic and amplified understanding of the world around them. In this unique application of these technologies, those who are hard of seeing and/or dual sensory impaired receive a more genuine appreciation for texts.

## How we built it

As our project required two extremely different pieces, we decided to split up our resources in hopes of tackling both problems at the same time. Regardless, we needed to set firm goals and plan out our required resources or timeline which helped us stay on schedule and formulate a final product that fully utilized our expertise.
In terms of hardware, we were somewhat limited for the first half of the hackathon as we had to purchase many of our materials and were unable to complete much of this work till later. We started by identifying a potential circuit design and creating a rigid structure to house our components. From there we simply spent a large amount of time actually implementing our theoretical circuit and applying it to our housing model in addition to cleaning the whole product up.
For software, we mostly had problems with connecting each of the pieces after building them out. We first created an algorithm that could take a camera feed and produce a coherent string of text. This would then be run through an AI text to speech generator that could decipher any gibberish. Finally, these texts would be sent through to either be read out loud or be compared against a dictionary to create a binary code that would dictate the on/off states off our solenoids.
Finally, we prototyped our product and tested it to see what we could improve in our final implementation to both increase efficiency and decrease latency.

## Challenges we ran into

This project was extremely technical and ambitious which meant that it was plagued with difficulties. As a large portion of the project relied on its hardware and implementing complementary materials to formulate a cohesive product, there were countless problems throughout the building phase. We often had incompatible parts whether it be cables, Voltage output/input, or even sizing and scaling issues, we were constantly scrambling to alter, scavenge, and adapt materials for uncommon use cases. Even our main board didn’t produce enough power, leading to an unusual usage of a dehumidifier charger and balled up aluminum foil as a makeshift power bank. All of these mechanical complexities followed by a difficult software end of the project led to an innovative and reworked solution that maintained applicative efficiency. These modifications even continued just hours before the submission deadline when we revamped the entire physical end of our project to make use of newly acquired materials using a more efficient modeling technique. These last second improvements gave our product a more polished and adept edge, making a more impactful and satisfying design.
Software wise we also strove to uncover the underappreciated features from our various APIs and tools which often didn’t coincide with our team’s strengths. As we had to simultaneously build out an effective product while troubleshooting our software side, we often ran into incompetencies and struggles. Regardless, we were able to overcome these adversities and produce an impressive result.

## Accomplishments that we're proud of

We are proud that we were able to overcome the various difficulties that arose throughout our process and to still showcase the level of success that we did even given such a short timeframe. Our team came in with some members having never done a hackathon before and we made extremely ambitious goals that we were unsure we could uphold. However, we were able to effectively work as a team to develop a final product that clearly represents our initial intentions for the project.

## What we learned

As a result of the many cutting-edge sponsors and new technological constraints, our whole team was able to draw from new more effective tools to increase efficiency and quality of our product. Through our careful planning and consistent collaboration, we experienced the future of software and progressed in our intricate technical knowledge within our fields and across specializations. and Because of the cross discipline nature of this project. Additionally, we became more flexible with what materials we needed to build out our hardware applications and especially utilized new TTS technologies to amplify the impact of our projects. In the future, we intend to continue to develop these crucial skills that we obtained at Cal Hacks 11.0, working towards a more accessible future.

## What's next for Text to Dot

We would like to work on integrating a more refined design to the hardware component of our project. Unforeseen circumstances with the solenoid led to our final design needing to be adjusted beyond the design of the original model, which could be rectified in future iterations.
## Post-Event Update

We are extremely grateful to have been chosen as one of the 6 finalists. Thank you for hosting a memorable first hackathon QHacks!

## Inspiration

Our team wanted to create something that would be useful in today's world *while also aiming for the Best Use of Google Cloud Award*. Inspired by the theme, "The New Wave of Tech", we wanted to push boundaries of what a simple mobile app could do . . . and thus we made Lucid, an app with the ability to help the blind.

## What it does

As explained, Lucid aims to make life easier for the blind. It's main function is to use the **phone's camera** and **text-to-speech** to tell the user what objects are in front of them (**object recognition**) or what the text in front of them says (**text-recognition**). Ex: Grocery shopping, blind people can't tell if an apple is red or green! We also have incredible UI features built in like **physical/tactile feedback** on button clicks, **text-to-speech** to describe the activity that appears on the screen, **voice command startup**, **speech speed** customization, and many other features so that Lucid is perfectly curated for users without sight.

## How we built it

**Backend:** Using Google Cloud's Vision API, Machine Learning Kit (machine learning for mobile developers), Java, and Android Studio

**Frontend:** XML and Android Studio

## Challenges we ran into

Going into the Hackathon, we lost 2 members. So that left only me to build the app and my partner, Andy Ho, to develop the pitch/demo materials. In the end it all worked out and the thrill of the challenges really pushed us.

## Accomplishments that we're proud of

Creating and learning so much in such a short amount of time, especially after the challenges we both ran into.

## What we learned

We learned about Google Cloud and their resources. We also found it very interesting to learn how to make use of cameras and text-to-speech in Android Studio.

## What's next for Lucid

There are many things Lucid can improve on like it's accuracy on image recognition, more language support, make use of Google Cloud's translation API to convert text from one language to speech in another, and many more!

## Contact

Team Channel #2 (coldcabbage#8622 and greenbean537#6339)

Jason Truong
<https://www.linkedin.com/in/jasonn-truong>
<https://github.com/jasonntruong>

Andy Ho
<https://www.linkedin.com/in/andy-ho-34b62b198/>
<https://github.com/AndyHo537>