## Inspiration

The idea for Refrigerator Ramsay came from a common problem we all face: food waste. So many times, we forget about ingredients in the fridge, or we don't know how to use them, and they go bad. We wanted to create something that could help us not only use those ingredients but also inspire us to try new recipes and reduce food waste in the process.

## What it does

Refrigerator Ramsay is an AI-powered kitchen assistant that helps users make the most of their groceries. You can take a picture of your fridge or pantry, and the AI will identify the ingredients. Then, it suggests recipes based on what you have. It also has an interactive voice feature, so you can ask for cooking tips or get help with recipes in real-time.

## How we built it

We used **Google Gemini AI** for image recognition to identify ingredients from photos. and then, to come up with creative meal ideas. We also added **Hume EVI**, which is an emotionally intelligent voice AI, to make the experience more interactive. This lets users chat with the AI to get tips, learn new cooking techniques, or personalize recipes.

## Challenges we ran into

One challenge we faced was working with **Hume EVI** for the first time. It was really fun to explore, but there was definitely a learning curve. We had to figure out how to make the voice assistant not just smart but also engaging, making sure it could deliver cooking tips naturally and understand what users were asking. It took some trial and error to get the right balance between information and interaction.

Another challenge was training the **Google Gemini AI** to be as accurate as possible when recognizing ingredients. It wasn't easy to ensure the AI could reliably detect different foods, especially when items were stacked or grouped together in photos. We had to spend extra time fine-tuning the model so that it could handle a variety of lighting conditions and product packaging.

## Accomplishments that we're proud of

We're proud of creating a solution that not only helps reduce food waste but also makes cooking more fun. The AI's ability to suggest creative recipes from random ingredients is really exciting. We're also proud of how user-friendly (and funny) the voice assistant turned out to be, making it feel like you're cooking with a friend.

## What we learned

We learned a lot about how AI can be used to solve everyday problems, like reducing food waste. Working with **Hume EVI** taught us about building conversational AI, which was new for us. It was really fun, but there was definitely a challenge in figuring out how to make the voice interaction feel natural and helpful at the same time.

We also learned about the importance of **training AI models**, especially with the **Gemini AI**. Getting the image recognition to accurately identify different ingredients required us to experiment a lot with the data and train the model to work in a variety of environments. This taught us that accuracy is key when it comes to creating a seamless user experience.

## What's next for Refrigerator Ramsay

Next, we want to improve the AI’s ability to recognize even more ingredients and maybe even offer nutritional advice. We’re also thinking about adding features where the AI can help you plan meals for the whole week based on what you have. Eventually, we’d love to partner with grocery stores to suggest recipes based on store deals, helping users save money too!
# Omakase

*"I'll leave it up to you"*

## Inspiration

On numerous occasions, we have each found ourselves staring blankly into the fridge with no idea of what to make. Given some combination of ingredients, what type of good food can I make, and how?

## What It Does

We have built an app that recommends recipes based on the food that is in your fridge right now. Using Google Cloud Vision API and Food.com database we are able to detect the food that the user has in their fridge and recommend recipes that uses their ingredients.

## What We Learned

Most of the members in our group were inexperienced in mobile app development and backend. Through this hackathon, we learned a lot of new skills in Kotlin, HTTP requests, setting up a server, and more.

## How We Built It

We started with an Android application with access to the user’s phone camera. This app was created using Kotlin and XML. Android’s ViewModel Architecture and the X library were used. This application uses an HTTP PUT request to send the image to a Heroku server through a Flask web application. This server then leverages machine learning and food recognition from the Google Cloud Vision API to split the image up into multiple regions of interest. These images were then fed into the API again, to classify the objects in them into specific ingredients, while circumventing the API’s imposed query limits for ingredient recognition. We split up the image by shelves using an algorithm to detect more objects. A list of acceptable ingredients was obtained. Each ingredient was mapped to a numerical ID and a set of recipes for that ingredient was obtained. We then algorithmically intersected each set of recipes to get a final set of recipes that used the majority of the ingredients. These were then passed back to the phone through HTTP.

## What We Are Proud Of

We were able to gain skills in Kotlin, HTTP requests, servers, and using APIs. The moment that made us most proud was when we put an image of a fridge that had only salsa, hot sauce, and fruit, and the app provided us with three tasty looking recipes including a Caribbean black bean and fruit salad that uses oranges and salsa.

## Challenges You Faced

Our largest challenge came from creating a server and integrating the API endpoints for our Android app. We also had a challenge with the Google Vision API since it is only able to detect 10 objects at a time. To move past this limitation, we found a way to segment the fridge into its individual shelves. Each of these shelves were analysed one at a time, often increasing the number of potential ingredients by a factor of 4-5x. Configuring the Heroku server was also difficult.

## Whats Next

We have big plans for our app in the future. Some next steps we would like to implement is allowing users to include their dietary restriction and food preferences so we can better match the recommendation to the user. We also want to make this app available on smart fridges, currently fridges, like Samsung’s, have a function where the user inputs the expiry date of food in their fridge. This would allow us to make recommendations based on the soonest expiring foods.
## Inspiration

We love cooking and watching food videos. From the Great British Baking Show to Instagram reels, we are foodies in every way. However, with the 119 billion pounds of food that is wasted annually in the United States, we wanted to create a simple way to reduce waste and try out new recipes.

## What it does

lettuce enables users to create a food inventory using a mobile receipt scanner. It then alerts users when a product approaches its expiration date and prompts them to notify their network if they possess excess food they won't consume in time. In such cases, lettuce notifies fellow users in their network that they can collect the surplus item. Moreover, lettuce offers recipe exploration and automatically checks your pantry and any other food shared by your network's users before suggesting new purchases.

## How we built it

lettuce uses React and Bootstrap for its frontend and uses Firebase for the database, which stores information on all the different foods users have stored in their pantry. We also use a pre-trained image-to-text neural network that enables users to inventory their food by scanning their grocery receipts. We also developed an algorithm to parse receipt text to extract just the food from the receipt.

## Challenges we ran into

One big challenge was finding a way to map the receipt food text to the actual food item. Receipts often use annoying abbreviations for food, and we had to find databases that allow us to map the receipt item to the food item.

## Accomplishments that we're proud of

lettuce has a lot of work ahead of it, but we are proud of our idea and teamwork to create an initial prototype of an app that may contribute to something meaningful to us and the world at large.

## What we learned

We learned that there are many things to account for when it comes to sustainability, as we must balance accessibility and convenience with efficiency and efficacy. Not having food waste would be great, but it's not easy to finish everything in your pantry, and we hope that our app can help find a balance between the two.

## What's next for lettuce

We hope to improve our recipe suggestion algorithm as well as the estimates for when food expires. For example, a green banana will have a different expiration date compared to a ripe banana, and our scanner has a universal deadline for all bananas.