## Inspiration

* ### State Machines and Video Games

As computer science enthusiasts, we love learning about different computational models, and one model we’re currently learning about is the state machine. State machines are hardcoded in video games and are more predictable than typical AI neural networks, but we began to wonder if there is a more efficient way to generate conditions. Through our simulation, we’ve trained the state machine to do just that.
* ### Evolution Simulators

Evolution simulators are cool to look at and very informative in the ecology and biology fields, but they are a pain to make yourself. Their systems are very complex and have a lot of variables to account for. We wanted to know if there was a simpler way to make this simulation, so that you can test out the environment and rules of the world in an easier way. We can do this using a state machine!
* ### Biological Education

Everyone knows about Darwin's Theory of Evolution (or at least has heard of it). As we all know, evolution can take hundreds of years, but wouldn't it be cool to see it happen in real time? We made a simulation where you can observe natural selection and gene inheritance play in an ecosystem.

---

## What it does

Blobolution is a 2D ecosystem simulation that is primarily an evolving system. This evolution is achieved through changing state machines! A state machine represents the states of an object and the conditions for changing that state. This is often used in video games, and examples of our states include GoToFood and AvoidEnemy.

### Features

* Birth System: asexual reproduction
* Mutation System: creates new combinations of traits in offspring
* Natural Selection: organisms who can't obtain enough energy die off
* Nutrient Recycling: energy of dead organisms is redeposited in the ecosystem as food
* Food Chain: traits demonstrate categorization of herbivores/carnivores and predators/prey
* Predatory Traits: more prominent when food is scarce (pack hunting)

---

## How we built it

The overall process was divided into 4 phases with tasks divided amongst 3 team members. We developed/trained state machine conditions rather than developing a neural network. When generating conditions, we compared values to a threshold to determine organism activity and state. This created a more predictable system compared to typical AI neural networks that tend to be too complex for small systems.

Another key foundation of our program is the genetic algorithm. We created unique genes to create various combinations of traits within our generated organisms. There is also an inherited mutation system in which traits merge and create new organisms that behave in relation to their parents. These relationships are depicted through colours and their proximity in tone. Organisms with similar traits are of similar colour!

---

## Challenges we ran into

We found issues establishing dominant species within our ecosystem. Initially, organisms classified as herbivores (consuming only food/nutrients) were capable of retaliating against carnivores (predators of other organisms). To address this, we only allowed hunting capabilities in the GoToEnemy state, preventing herbivore attacks on carnivores. However, herbivores still thrived enough that a dominant species remained unclear. Additionally, all organisms have the same walk speed, so carnivores would chase prey out of the map, eventually losing energy and dying.

To address these issues, we introduced a system where each organism travels back to its sleep spot if they can’t find their target, whether food or prey. This heavily advantaged carnivores because prey travelled to the center of the map if they were too far from any food, which is the direction of the predators.

Our latest solution was to introduce warping at the edges of the map to create a more balanced system in which prey utilize a state machine when necessary. The results showed more evidence of the "fleeing" state now that prey can move away from carnivores infinitely because they are teleported from one edge of the map to the opposite side. This system also gives carnivores a chance by changing their targets often, since targets are set based on proximity. Additionally, all organisms are confined to the area of the map, preventing “endless” chasing that kills both predators and prey from loss of energy.

---

## Accomplishments that we're proud of

We have accomplished so much throughout this hackathon. We are proud that:

* We finished our project efficiently and in a timely manner
* We worked together as a cohesive and effective team
* We worked through the challenges that we faced with success
* We made an evolution simulation that demonstrates how organisms interact in an ecosystem

---

## What we learned

Having a clear plan and outlining division of labour before beginning the technical portion of the project is incredibly helpful in ensuring an efficient development process. The time constraints of a hackathon easily induce stress amongst even the strongest teams, so assigning tasks before the pressure sets in is key to level-headed and strategic thinking.

---

## What's next for Blobolution

### Modifications

Our ecosystem simulation can continue to be expanded upon, including extensions such as:

* Time lapse: to increase the speed of environment activities. This way users can analyze more complex and unique gene combinations.
* Sexual Reproduction: experimenting with fish reproduction (egg laying) to reduce grouping
* More Diverse Traits: digestion bias, weight, etc.
* A User Interface: for easy analysis of genes and an organism’s inner workings
* Increased Organism Variety: more states, traits, conditions and improving the mutation system
* Game System: potentially implementing AI boss
* Comparison: analyze the effectiveness of our state machine method in comparison to neural network training

### Future Applications

* Blobolution can be applied in educational settings to demonstrate the interactions of organisms within an ecosystem. This simulation includes applicable features such as family trees, a reproductive system, and a food chain.
* Blobolution can also be used in an experimental research context to explore how genetic changes affect an organism and its mutated offspring.
* The fact that Blobolution is more generative than hardcoded is an improved way to implement state machines in game development. This method is more efficient and still preserves the quality of being more predictable that traditional AI neural networks.

---
## What it does

Our site allows a user to take a picture of their outfit and receive AI-generated feedback on their style, color palette, and cohesiveness. While we are not your typical education tool, we think that dressing well has a massive boost on your confidence, your presence, and your ability to talk about yourself! Our goal is to give everyone a chance to be their most styling self.

In addition to hyping up the best of your outfit, we recommend relevant pieces to incorporate into your style as you explore defining your visual voice. Discovering fashion should be exciting, not intimidating -- and with the help of LLM's, **Drip or Drown** does exactly that!

## How we built it

During the course of the weekend, we experimented with many different approaches but ultimately settled on the following architecture ([click here for full res](https://i.ibb.co/8rBwgqB/architecture2.png)):
![architecture](https://i.ibb.co/NL9kZvy/architecture.png). Here's some of the salient elements:

1. Depth Perception
When an image is uploaded, we use computer vision to remove noise. We use the MIDAS depth perception model to separate the background from the foreground, create a separate image only of the person and their fit!
2. Visual Transformation Model (Q&A):
We used the ViLT QnA model to query the de-noised image about attributes about the outfit — extracting information about what they're wearing and details in a recursive fashion.
3. In-Context Learning Fashion Critique Model
Finally, we use Large Language Models to generate in-depth feedback for your fit using our description. We also use a model to rate it on a scale of 1-10 and another to categorize it into one of many "aura"s, so that you can have a well-rounded understanding of your own style!

## Challenges we ran into

1. Figuring out deployment was particularly hard. Given the many moving parts, our project is hosted on multiple platforms that all interact with each other.
2. Using non-production research models like ViLT meant that our performance for the API calls was quite abysmal, and we had to get pretty clever about parallelization and early-stopping algorithms within our call structure.
3. We spent a lot of time adding juice to the project! Making it fun to use was a big goal of ours, which was often easier said than done :)

## Accomplishments that we're proud of

1. The UI! We think it looks pretty great – one of our teammates mocked it all up in Figma, and then we spent most of Sunday night making components together. We really wanted to make our project fun to use, and I think we accomplished that.
2. Using Multi-modal AI! It's one of the biggest unsolved problems in AI right now. How do you use multiple forms of input — image *and* text — together? I think we came up with a pretty clever solution that works quite well, and is pretty interpretable as well!

## What we learned

1. A lot about many technologies! Like Flask, PythonAnywhere, AWS, Heroku, Vercel, generating particles, CSS, Visual Transformer models, GPT fine-tuning, image progressing, classification algorithms, and more! This project spans many different domains, and it was pretty fun to pick up skills along the way.
2. The need for patience! For the longest time, we would have "blocker" bugs that would prevent us from deploying or developing further. We pushed ahead, and every time we handled those, the emergent abilities of the system surprised us as well.
3. And of course, that having fun once in a while is important. We did some of our best work when we were all singing to pop songs together at 11PM.

## What's next for Drip or Drown

1. Improving suggestion quality: We think we can push this even further! While our current image -> text algorithm is clever, we think we could make it even smarter by using a shared embedding space between images and text. This could capture attributes of the image our QnA model could not!
2. Follow-ups and conversation: We'd love for you to be able to ask the model questions about your fit! "Why does a white belt work better?" "What do you think about this leather jacket with that shirt?"
3. Suggestions! Finally, from all the feedback, we'd love for the model to be able to suggest fits as well. "You'd look great in a green croptop for this casual event!" We hope that Drip is the AI assistant to help you achieve your most fashionable self.
## Inspiration

The day to day process of navigating to websites and then adding products to the cart is time-consuming. So We thought of making a voice-enabled shopping assistant which can search and add products to the cart just with a voice command.

## What it does

It can search and add products to the cart just with a voice command. It is also capable of performing a day-to-day activity such as playing music, opening applications, etc

## How I built it

We have built it in python using the selenium library and performing web automation

## Challenges I ran into

Web automation was really tough

## Accomplishments that I'm proud of

The final product which performs on voice command is an accomplishment.

## What I learned

I have learned many new skills like web automation, python libraries, etc

## What's next for Voice Shop - Voice-enabled Shopping Assistant