## Inspiration

We believe current reCAPTCHA v.3 has few problems. First, it is actually hard to prove myself to be not robot. It is because Machine Learning is advancing everyday, and ImageToText's (Computer Vision) accuracy is also skyrocketing. Thus, CAPTCHA question images have to be more difficult and vague. Second, the dataset used for current CAPTCHA is limited. It becomes predictable as it's repeating its questions or images (All of you should have answered "check all the images with traffic lights"). In this regard, several research paper has been published through Black Hat using Machine learning models to break CAPTCHA.

## What it does

Therefore, we decided to build a CAPTCHA system that would generate a totally non-sensical picture, and making humans to select the description for that AI-created photo of something 'weird'. As it will be an image of something that is non-existent in this world, machine learning models like ImageToText will have to idea what the matching prompt would be. However, it will be very clear for human even though the images might not be 100% accurate of the description, it's obvious to tell which prompt the AI try to draw. Also, it will randomly create image from scratch every time, we don't need a database having thousands of photos and prompts. Therefore, we will be able to have non-repeating 'im not a robot' question every single time -> No pattern, or training data for malicious programs.
Very easy and fun 'Im not a robot' challenge.

## How we built it

We used AI-painting model called 'Stable Diffusion', which takes a prompt as an input, and creates an image of the prompt. The key of our CAPTCHA is that the prompt that we feed in to this model is absurd and non-existent in real world. We used NLP APIs provided by Cohere in order to generate this prompts. Firstly, we gathered 4,000 English sentences and clustered them to groups based on the similarity of topics using Cohere's embed model. Then, from each clusters, we extracted on key words and using that keywords generated a full sentence prompt using Cohere's generate model. And with that prompt, we created an image using stable diffusion.

## Challenges we ran into

As stable-diffusion is a heavy computation and for sure needed GPU power, we needed to use a cloud GPU. However, cloud GPU that we used from paperspace had its own firewall, which prevented us to deploy server from the environment that we were running tests.

## Accomplishments that we're proud of

We incorporated several modern machine learning techniques to tackle a real world problem and suggested a possible solution. CAPTCHA is especially a security protocol that basically everyone who uses internet encounters. By making it less-annoying and safer, we think it could have a positive impact in a large scale, and are proud of that.

## What we learned

We learned about usability of Cohere APIs and stable diffusion. Also learned a lot about computer vision and ImageToText model, a possible threat model for all CAPTCHA versions. Additionally, we learned a lot about how to open a server and sending arguments in real-time.

## What's next for IM NOT A ROBOT - CAPTCHA v.4

As not everyone can run stable diffusion on their local computer, we need to create a server, which the server does the calculation and creation for the prompt and image.
## Inspiration

A chatbot is often described as one of the most advanced and promising expressions of interaction between humans and machines. For this reason we wanted to create one in order to become affiliated with Natural Language Processing and Deep-Learning through neural networks.

Due to the current pandemic, we are truly living in an unprecedented time. As the virus' spread continues, it is important for all citizens to stay educated and informed on the pandemic. So, we decided to give back to communities by designing a chatbot named Rona who a user can talk to, and get latest information regarding COVID-19.

(This bot is designed to function similarly to ones used on websites for companies such as Amazon or Microsoft, in which users can interact with the bot to ask questions they would normally ask to a customer service member, although through the power of AI and deep learning, the bot can answer these questions for the customer on it's own)

## What it does

Rona answers questions the user has regarding COVID-19.

More specifically, the training data we fed into our feed-forward neural network to train Rona falls under 5 categories:

* Deaths from COVID-19
* Symptoms of COVID-19
* Current Cases of COVID-19
* Medicines/Vaccines
* New Technology/Start-up Companies working to fight coronavirus

We also added three more categories of data for Rona to learn, those being greetings, thanks and goodbyes, so the user can have a conversation with Rona which is more human-like.

## How we built it

First, we had to create my training data. Commonly referred to as 'intentions', the data we used to train Rona consisted of different phrases that a user could potentially ask. We split up all of my intentions into 7 categories, which we listed above, and these were called 'tags'. Under our sub-branch of tags, we would provide Rona several phrases the user could ask about that tag, and also gave it responses to choose from to answer questions related to that tag. Once the intentions were made, we put this data in a json file for easy access in the rest of the project.

Second, we had to use 3 artificial-intelligence, natural language processing, techniques to process the data, before it was fed into our training model. These were 'bag-of-words', 'tokenization' and 'stemming'. First, bag-of-words is a process which took a phrase, which were all listed under the tags, and created an array of all the words in that phrase, making sure there are no repeats of any words. This array was assigned to an x-variable. A second y-variable delineated which tag this bag-of-words belonged to. After these bags-of-words were created, tokenization was applied through each bag-of-words and split them up even further into individual words, special characters (like @,#,$,etc.) and punctuation. Finally, stemming created a crude heuristic, i.e. it chopped off the ending suffixes of the words (organize and organizes both becomes organ), and replaced the array again with these new elements. These three steps were necessary, because the training model is much more effective when the data is pre-processed in this way, it's most fundamental form.

Next, we made the actual training model. This model was a feed-forward neural network with 2 hidden layers. The first step was to create what are called hyper-parameters, which is a standard procedure for all neural networks. These are variables that can be adjusted by the user to change how accurate you want your data to be. Next, the network began with 3 layers which were linear, and these were the layers which inputted the data which was pre-processed earlier. After, these were passed on into what are called activation functions. Activation functions output a small value for small inputs, and a larger value if its inputs exceed a threshold. If the inputs are large enough, the activation function "fires", otherwise it does nothing. In other words, an activation function is like a gate that checks that an incoming value is greater than a critical number.

The training was completed, and the final saved model was saved into a 'data.pth' file using pytorch's save method.

## Challenges we ran into

The most obvious challenge was simply time constraints. We spent most of our time trying to make sure the training model was efficient, and had to search up several different articles and tutorials on the correct methodology and API's to use. Numpy and pytorch were the best ones.

## Accomplishments that we're proud of

This was our first deep-learning project so we are very proud of completing at least the basic prototype. Although we were aware of NLP techniques such as stemming and tokenization, this is our first time actually implementing them in action. We have created basic neural nets in the past, but also never a feed-forward one which provides an entire model as its output.

## What we learned

We learned a lot about deep learning, neural nets, and how AI is trained for communication in general. This was a big step up for us in Machine Learning.

## What's next for Rona: Deep Learning Chatbot for COVID-19

We will definitely improve on this in the future by updating the model, providing a lot more types of questions/data related to COVID-19 for Rona to be trained on, and potentially creating a complete service or platform for users to interact with Rona easily.
## Inspiration

The inspiration for this project came from the contour design on the gift bag design of UofTHacks-X. A contour is created by removing a dimension from a 3D object, which got us thinking, is it possible to insert a dimension into an image to make it 3D?

Dall E AI wowed the world with text-to-image generation earlier this year, and in this project, we want to break the bound of 2 dimensions and generate a 3D object using text as the latest breakthrough in neural radiance field (NeRF) and image generation from text made this possible.

## What it does

Our site uses a text-based input to create a 3D textured mesh using stable diffusion techniques and uploads it to Google Cloud Storage for fast and secure storage. We are also open to the idea of trading these AI-generated objects as NFTs in the future.

## How we built it

The full-stack web app was developed using node.js and JavaScript. The JavaScript backend calls accept user inputs and augment them using cohere's language generation algorithm to create a more detailed prompt for the object generation algorithm to generate more specific objects. After the object is generated, the mesh and the texture files are rendered using WebGL and three.js so the user can manipulate it directly. Furthermore, the object is uploaded to Google Cloud Storage with a URL and can be downloaded by the users.

The 3D object generation algorithm was implemented in Python and CUDA based on dreamfusion's text-to-AI algorithm. The network uses stable diffusion techniques to generate multiple 2D images based on text input and fuses these 2D text images using NeRF (neural radiance field) to create a 3D object.

## Challenges we ran into

We encountered numerous challenges with running the 3D object generator because the stable diffusion algorithm requires a humongous amount of processing required to turn a blurry fuzzy point cloud into an object. For example, the RTX 3090 GPU with 24GB of VRAM needed to run for 45 minutes straight to generate a mesh like the hamburger in high resolution which takes too long for the demo. So we had to tweak the model parameters such as learning rate and iterations to find the best balance between time and quality to keep the generation time under 15 minutes.

## Accomplishments that we're proud of

We successfully learned and connected many new technologies in this project, from stable diffusion for object generation, to object rendering using three.js and WebGL, to decentralized storage and language generation models, we came a long way.

## What we learned

1. Stable diffusion for object generation and how NeRF works
2. Web-based rendering using three.js and WebGL
3. Cohere language generation model

## What's next for Fuzzy Dreams

Buy and sale AI-generated NFTs and metaverse VR integration