## Inspiration

When we heard about using food as a means of love and connection from Otsuka x VALUENEX’s Opening Ceremony presentation, our team was instantly inspired to create something that would connect Asian American Gen Z with our cultural roots and immigrant parents. Recently, there has been a surge of instant Asian food in American grocery stores. However, the love that exudes out of our mother’s piping hot dishes is irreplaceable, which is why it’s important for us, the loneliest demographic in the U.S., to cherish our immigrant parents’ traditional recipes. As Asian American Gen Z ourselves, we often fear losing out on beloved cultural dishes, as our parents have recipes ingrained in them out of years of repetition and thus, neglected documenting these precious recipes. As a result, many of us don’t have access to recreating these traditional dishes, so we wanted to create a web application that encourages sharing of traditional, cultural recipes from our immigrant parents to Asian American Gen Z. We hope that this will reinforce cross-generational relationships, alleviate feelings of disconnect and loneliness (especially in immigrant families), and preserve memories and traditions.

## What it does

Through this web application, users have the option to browse through previews of traditional Asian recipes, posted by Asian or Asian American parents, featured on the landing page. If choosing to browse through, users can filter (by culture) through recipes to get closer to finding their perfect dish that reminds them of home. In the previews of the dishes, users will find the difficulty of the dish (via the number of knives – greater is more difficult), the cultural type of dish, and will also have the option to favorite/save a dish. Once they click on the preview of a dish, they will be greeted by an expanded version of the recipe, featuring the name and image of the dish, ingredients, and instructions on how to prepare and cook this dish. For users that want to add recipes to *yumma*, they can utilize a modal box and input various details about the dish. Additionally, users can also supplement their recipes with stories about the meaning behind each dish, sparking warm memories that will last forever.

## How we built it

We built *yumma* using ReactJS as our frontend, Convex as our backend (made easy!), Material UI for the modal component, CSS for styling, GitHub to manage our version set, a lot of helpful tips and guidance from mentors and sponsors (♡), a lot of hydration from Pocari Sweat (♡), and a lot of love from puppies (♡).

## Challenges we ran into

Since we were all relatively beginners in programming, we initially struggled with simply being able to bring our ideas to life through successful, bug-free implementation. We turned to a lot of experienced React mentors and sponsors (shoutout to Convex) for assistance in debugging. We truly believe that learning from such experienced and friendly individuals was one of the biggest and most valuable takeaways from this hackathon. We additionally struggled with styling because we were incredibly ambitious with our design and wanted to create a high-fidelity functioning app, however HTML/CSS styling can take large amounts of time when you barely know what a flex box is. Additionally, we also struggled heavily with getting our app to function due to one of its main features being in a popup menu (Modal from material UI). We worked around this by creating an extra button in order for us to accomplish the functionality we needed.

## Accomplishments that we're proud of

This is all of our first hackathon! All of us also only recently started getting into app development, and each has around a year or less of experience–so this was kind of a big deal to each of us. We were excitedly anticipating the challenge of starting something new from the ground up. While we were not expecting to even be able to submit a working app, we ended up accomplishing some of our key functionality and creating high fidelity designs. Not only that, but each and every one of us got to explore interests we didn’t even know we had. We are not only proud of our hard work in actually making this app come to fruition, but that we were all so open to putting ourselves out of our comfort zone and realizing our passions for these new endeavors. We tried new tools, practiced new skills, and pushed our necks to the most physical strain they could handle. Another accomplishment that we were proud of is simply the fact that we never gave up. It could have been very easy to shut our laptops and run around the Main Quadrangle, but our personal ties and passion for this project kept us going.

## What we learned

On the technical side, Erin and Kaylee learned how to use Convex for the first time (woo!) and learned how to work with components they never knew could exist, while Megan tried her hand for the first time at React and CSS while coming up with some stellar wireframes. Galen was a double threat, going back to her roots as a designer while helping us develop our display component. Beyond those skills, our team was able to connect with some of the company sponsors and reinvigorate our passions on why we chose to go down the path of technology and development in the first place. We also learned more about ourselves–our interests, our strengths, and our ability to connect with each other through this unique struggle.

## What's next for yumma

Adding the option to upload private recipes that can only be visible to you and any other user you invite to view it (so that your Ba Ngoai–grandma’s—recipes stay a family secret!)

Adding more dropdown features to the input fields so that some will be easier and quicker to use

A messaging feature where you can talk to other users and connect with them, so that cooking meetups can happen and you can share this part of your identity with others

Allowing users to upload photos of what they make from recipes they make and post them, where the most recent of photos for each recipe will be displayed as part of a carousel on each recipe component.

An ingredients list that users can edit to keep track of things they want to grocery shop for while browsing
## Inspiration

We love to cook for and eat with our friends and family! Sometimes though, we need to accommodate certain dietary restrictions to make sure everyone can enjoy the food. What changes need to be made? Will it be a huge change? Will it still taste good? So much research and thought goes into this, and we all felt as though an easy resource was needed to help ease the planning of our cooking and make sure everyone can safely enjoy our tasty recipes!

## What is Let Them Cook?

First, a user selects the specific dietary restrictions they want to accommodate. Next, a user will copy the list of ingredients that are normally used in their recipe into a text prompt. Then, with the push of a button, our app cooks up a modified, just as tasty, and perfectly accommodating recipe, along with expert commentary from our "chef"! Our "chef" also provides suggestions for other similar recipes which also accomidate the user's specific needs.

## How was Let Them Cook built?

We used React to build up our user interface. On the main page, we implemented a rich text editor from TinyMCE to serve as our text input, allong with various applicable plugins to make the user input experience as seamless as possible.

Our backend is Python based. We set up API responses using FastAPI. Once the front end posts the given recipe, our backend passes the recipe ingredients and dietary restrictions into a fine-tuned large language model - specifically GPT4.

Our LLM had to be fine-tuned using a combination of provided context, hyper-parameter adjustment, and prompt engineering. We modified its responses with a focus on both dietary restrictions knowledge and specific output formatting.

The prompt engineering concepts we employed to receive the most optimal outputs were n-shot prompting, chain-of-thought (CoT) prompting, and generated knowledge prompting.

## UI/UX

### User Personas

![User Persona 1](https://lh3.googleusercontent.com/pw/ADCreHcPNSKWnt37BYEZkkTys193lo41ZemedLwl4tbr_OBXvYgZkLISUzuuXeuQBXyZBrlIye-u2maW5RcPK9RdPa3hI1RFZiEhHG1iuRirJkrtrvHZsbM=w2400)

*We build some user personas to help us better understand what needs our application could fulfil*

### User Flow

![User Flow](https://lh3.googleusercontent.com/pw/ADCreHeMI5gMUYaxxQCC7X-nvrs7085m0Z8Kjxszj53RVT6VPYqkCpHLgLr5LtO1HWIft0k7YL53qIzKE73q4V_CStSSFpRVsMLKK2FrH1BLIXmepZYIWTs=w2400)

*The user flow was made to help us determine the necessary functionality we wanted to implement to make this application useful*

### Lo-fi Prototypes

![Lofi Mockups](https://lh3.googleusercontent.com/pw/ADCreHf-E8p6OXBlCB1x_t78_ls43jR8Uh709LQhLrP1FmjP9sKSyEiVmcC7VqtnXrfiDeBlNcjHgWssIUE1DojQa-LxUC4EyDARPpCgGRKMbbH50oKTLR0=w2400)

*These lo-fi mockups were made to determine what layout we would present to the user to use our primary functionality*

### Hi-fi Prototypes

![hifi mockups 1](https://lh3.googleusercontent.com/pw/ADCreHcIY-iKuCfu-RLqIQo2W5Sav5xEPuJ8IRXId-_iDcXcgbSousCf0B6EKWJki3pCPREPyH0bDJbCXhbbZN4TcBp42B8pRRTfzIGj3k3pk6wyhrGayVY=w2400) ![hifi mockups 2](https://lh3.googleusercontent.com/pw/ADCreHffaIev4mI7swuOQ_UHkJxOvUI_FMsDuHASNRitEiE_006SQLxXEeAaArWyWc0mPNbyTMjbknn1VwiU3AV-OL1hGGnRPYnDjNXRz7Z3uS6Zr8hypis=w2400)

*Here we finalized the styling choice of a blue and yellow gradient, and we started planning for incorporating our extra feature as well - the recipe recomendations*

## Engineering

![Technologies](https://lh3.googleusercontent.com/pw/ADCreHejCSB2jA2J6vTYG1Dnc0HsXujeAYkax8eLmlSe2n3uLq9t-FsR0f9RhbcwuA-ZcvrdrCIoVieI26yjy-LoJeHT2xQp0BfRgNQzan4yEMZzaWtRZZg=w2400)
Frontend: React, JS, HTML, CSS, TinyMCE, Vite

Backend: FastAPI, Python

LLM: GPT4

Database: Zilliz

Hosting: Vercel (frontend), Render (backend)

## Challenges we ran into

### Frontend Challenges

Our recipe modification service is particularly sensitive to the format of the user-provided ingredients and dietary restrictions. This put the responsibility of vetting user input onto the frontend. We had to design multiple mechanisms to sanitize inputs before sending them to our API for further pre-processing. However, we also wanted to make sure that recipes were still readable by the humans who inputted them. Using the TinyMCE editor solved this problem effortlessly as it allowed us to display text in the format it was pasted, while simultaneously allowing our application to access a "raw", unformatted version of the text.

To display our modified recipe, we had to brainstorm the best ways to highlight any substitutions we made. We tried multiple different approaches including some pre-built components online. In the end, we decided to build our own custom component to render substitutions from the formatting returned by the backend.

We also had to design a user flow that would provide feedback while users wait for a response from our backend. This materialized in the form of an interim loading screen with a moving GIF indicating that our application had not timed out. This loading screen is dynamic, and automatically re-routes users to the relevant pages upon hearing back from our API.

### Backend Challenges

The biggest challenge we ran into was selecting a LLM that could produce consistent results based off different input recipes and prompt engineering. We started out with Together.AI, but found that it was inconsistent in formatting and generating valid allergy substitutions. After trying out other open-source LLMs, we found that they also produced undesirable results. Eventually, we compromised with GPT-4, which could produce the results we wanted after some prompt engineering; however, it it is not a free service.

Another challenge was with the database. After implementing the schema and functionality, we realized that we partitioned our design off of the incorrect data field. To finish our project on time, we had to store more values into our database in order for similarity search to still be implemented.

![interactions](https://lh3.googleusercontent.com/pw/ADCreHdDYpTiJoG9bTREKFJywBYrQoztexD9RXPK9jg1vNa0d8Ud9s7HAjRCnJZQodY2InKnEQyEF-iJIaexRGMlXAqdTtMLD8VIsHlFSNITrU3Ik0C2YKA=w2400)

## Takeaways

### Accomplishments that we're proud of

From coming in with no knowledge, we were able to build a full stack web applications making use of the latest offerings in the LLM space. We experimented with prompt engineering, vector databases, similarity search, UI/UX design, and more to create a polished product. Not only are we able to demonstrate our learnings through our own devices, but we are also able to share them out with the world by deploying our application.

**All that said**, our proudest accomplishment was creating a service which can provide significant help to many in a common everyday experience: cooking and enjoying food with friends and family.

### What we learned

For some of us on the team, this entire project was built in technologies that were unfamiliar. Some of us had little experience with React or FastAPI so that was something the more experienced members got to teach on the job.

One of the concepts we spent the most time learning was about prompt engineering.

We also learned about the importance of branching on our repository as we had to build 3 different components to our project all at the same time on the same application.

Lastly, we spent a good chunk of time learning how to implement and improve our similarity search.

### What's next for Let Them Cook

We're very satisfied with the MVP we built this weekend, but we know there is far more work to be done.

First, we would like to deploy our recipe similarity service currently working on a local environment to our production environment. We would also like to incorporate a ranking system that will allow our LLM to take in crowdsourced user feedback in generating recipe substitutions.

Additionally, we would like to enhance our recipe substitution service to make use of recipe steps rather than solely *ingredients*. We believe that the added context of how ingredients come together will result in even higher quality substitutions.

Finally, we hope to add an option for users to directly insert a recipe URL rather than copy-and-pasting the ingredients. We would write another service to scrape the site and extract the information a user would previously paste.
## Inspiration

Ordering delivery and eating out is a major aspect of our social lives. But when healthy eating and dieting comes into play it interferes with our ability to eat out and hangout with friends. With a wave of fitness hitting our generation as a storm we have to preserve our social relationships while allowing these health conscious people to feel at peace with their dieting plans. With NutroPNG, we enable these differences to be settled once in for all by allowing health freaks to keep up with their diet plans while still making restaurant eating possible.

## What it does

The user has the option to take a picture or upload their own picture using our the front end of our web application. With this input the backend detects the foods in the photo and labels them through AI image processing using Google Vision API. Finally with CalorieNinja API, these labels are sent to a remote database where we match up the labels to generate the nutritional contents of the food and we display these contents to our users in an interactive manner.

## How we built it

Frontend: Vue.js, tailwindCSS
Backend: Python Flask, Google Vision API, CalorieNinja API

## Challenges we ran into

As we are many first-year students, learning while developing a product within 24h is a big challenge.

## Accomplishments that we're proud of

We are proud to implement AI in a capacity to assist people in their daily lives. And to hopefully allow this idea to improve peoples relationships and social lives while still maintaining their goals.

## What we learned

As most of our team are first-year students with minimal experience, we've leveraged our strengths to collaborate together. As well, we learned to use the Google Vision API with cameras, and we are now able to do even more.

## What's next for McHacks

* Calculate sum of calories, etc.
* Use image processing to estimate serving sizes
* Implement technology into prevalent nutrition trackers, i.e Lifesum, MyPlate, etc.
* Collaborate with local restaurant businesses