## Inspiration

Often times we’ve seen people post live streams on social media, use video calling for a more personal touch to a conversation, and engage in activities which require extensive expression of emotion through language. However, with over 430 million people around the world having hearing disabilities and less than a fourth of people fluent in sign language, our team was sought to bridge this significant, yet ironically, unattended problem.

## What it does

ConnexionAI.tech is a web application that converts video input of sign language to three most popular languages English, Spanish, and French. The application also has in built capability of converting speech to text and also translate various languages to English - Text . In the future, ConnexionAI can be deployed in video chats and live streams to caption people with speaking disabilities to caption sign language and it could be used as a live ASL to text convertor.

## How I built it

The machine learning model was trained on google cloud’s autoML platform, the web framework we’re using is flask, the website was built in HTML5, CSS, JS, and the project integration was in python.

## Challenges I ran into

Our biggest challenge was to find a reliable dataset and deciding the hyper parameters for training the model under such a time constraint. The other challenge we ran into was integrating the video format in the web browser and taking inputs to backend and pushing outputs to the browser again.

## Accomplishments that I'm proud of

We are proud of having the opportunity to solve a problem that affects the lives of millions of people in a fun learning environment!

## What I learned

All of us worked with something that we had worked with before. Using google cloud for the first time was fascinating and the idea of connecting millions of dots really inspired us to pull through a working model.

## What's next for Connexion

Future possibilities include implementing connexion to livestreams on Facebook and videos on YouTube to caption sign language for all audiences.
## Inspiration

Our inspiration was to provide a robust and user-friendly financial platform. After hours of brainstorming, we decided to create a decentralized mutual fund on mobile devices. We also wanted to explore new technologies whilst creating a product that has several use cases of social impact. Our team used React Native and Smart Contracts along with Celo's SDK to discover BlockChain and the conglomerate use cases associated with these technologies. This includes group insurance, financial literacy, personal investment.

## What it does

Allows users in shared communities to pool their funds and use our platform to easily invest in different stock and companies for which they are passionate about with a decreased/shared risk.

## How we built it

* Smart Contract for the transfer of funds on the blockchain made using Solidity
* A robust backend and authentication system made using node.js, express.js, and MongoDB.
* Elegant front end made with react-native and Celo's SDK.

## Challenges we ran into

Unfamiliar with the tech stack used to create this project and the BlockChain technology.

## What we learned

We learned the many new languages and frameworks used. This includes building cross-platform mobile apps on react-native, the underlying principles of BlockChain technology such as smart contracts, and decentralized apps.

## What's next for *PoolNVest*

Expanding our API to select low-risk stocks and allows the community to vote upon where to invest the funds.
Refine and improve the proof of concept into a marketable MVP and tailor the UI towards the specific use cases as mentioned above.
## Inspiration

Just in the United States, 11 million people are deaf, of which half a million use ASL natively. For people who are deaf to communicate with those who are hearing, and therefore most likely not fluent in sign language, they’re constricted to slower methods of communication, such as writing. Similarly, for people who don’t know sign language to immediately be able to communicate comfortably with someone who is deaf, the only options are writing or lip-reading, which has been proven to only be 30-40% effective. By creating Gesture Genie, we aim to make the day to day lives of both the deaf and hearing communities significantly more tension-free.

## What it does

Two Way Sign Language Translator that can translate voice into visual sign language and can translate visual sign language into speech. We use computer vision algorithms to recognize the sign language gestures and translate into text.

## How we built it

Sign Gestures to Text

* We used OpenCV to to capture video from the user which we processed into
* We used Google MediaPipe API to detect all 21 joints in the user's hands
* Trained neural networks to detect and classify ASL gestures

<https://youtu.be/3gNQDp1nM_w>

Voice to Sign Language Gestures

* Voice to text translation
* Text to words/fingerspelling
* By web scraping online ASL dictionaries we can show videos for all ASL through gestures or fingerspelling
* Merging videos of individual sign language words/gestures

<https://youtu.be/mBUhU8b3gK0?si=eIEOiOFE0mahXCWI>

## Challenges we ran into

The biggest challenge that we faced was incorporating the backend into the React/node.js frontend. We use OpenCV for the backend, and we needed to ensure that the camera's output is shown on the front end. We were able to solve this problem by using iFrames in order to integrate the local backend into our combined webpage.

## Accomplishments that we're proud of

We are most proud of being able to recognize action-based sign language words rather than simply relying on fingerspelling to understand the sign language that we input. Additionally, we were able to integrate these videos from our backend into the front end seamlessly using Flask APIs and iFrames.

## What we learned

## What's next for Gesture Genie

Next, we hope to be able to incorporate more phrases/action based gestures, scaling our product to be able to work seamlessly in the day-to-day lives of those who are hard of hearing. We also aim to use more LLMs to better translate the text, recognizing voice and facial emotions as well.