## Inspiration

**Powerful semantic search for your life does not currently exist.**

Google and ChatGPT have brought the world’s information to our fingertips, yet our personal search engines — Spotlight on Mac, and search on iOS or Android — are insufficient.

Google Assistant and Siri tried to solve these problems by allowing us to search and perform tasks with just our voice, yet their use remains limited to a narrow range of tasks. **Recent advancement in large language models has enabled a significant transformation in what's possible with our devices.**

## What it does

That's why we made Best AI Buddy, or BAIB for short.

**BAIB (pronounced "babe") is designed to seamlessly answer natural language queries about your life.** BAIB builds an index of your personal data — text messages, emails, photos, among others — and runs a search pipeline on top of that data to answer questions.

For example, you can ask BAIB to give you gift recommendations for a close friend. BAIB looks pertinent interactions you've had with that friend and generates gift ideas based on their hobbies, interests, and personality. To support its recommendations, **BAIB cites parts of past text conversations you had with that friend.**

Or you can ask BAIB to tell you about what happened the last time you went skiing with friends. BAIB intelligently combines information from the ski group chat, AirBnB booking information from email, and your Google Photos to provide you a beautiful synopsis of your recent adventure.

**BAIB understands “hidden deadlines”** — that form you need to fill out by Friday or that internship decision deadline due next week — and keeps track of them for you, sending you notifications as these “hidden deadlines” approach.

**Privacy is an essential concern.** BAIB currently only runs on M1+ Macs. We are working on running a full-fledged LLM on the Apple Neural Engine to ensure that all information and processing is kept on-device. We believe that this is the only future of BAIB that is both safe and maximally helpful.

## How we built it

Eventually, we plan to build a full-fledged desktop application, but for now we have built a prototype using the SvelteKit framework and the Skeleton.dev UI library. We use **Bun as our TypeScript runtime & toolkit.**

**Python backend.** Our backend is built in Python with FastAPI, using a few hacks (check out our GitHub) to connect to your Mac’s contacts and iMessage database. We use the Google API to connect to Gmail + photos.

**LLM-guided search.** A language model makes the decisions about what information should be retrieved — what keywords to search through different databases — and when to generate a response or continue accumulating more information. A beautiful, concise answer to a user query is often a result of many LLM prompts and aggregation events.

**Retrieval augmented generation.** We experimented with vector databases and context-window based RAG, finding the latter to be more effective.

**Notifications.** We have a series of “notepads” on which the LLM can jot down information, such as deadlines. We then later use a language model to generate notifications to ensure you don’t miss any crucial events.

## Challenges we ran into

**Speed.** LLM-guided search is inherently slow, bottlenecked by inference performance. We had a lot of difficulty filtering data before giving it to the LLM for summarization and reasoning in a way that maximizes flexibility while minimizing cost.

**Prompt engineering.** LLMs don’t do what you tell them, especially the smaller ones. Learning to deal with it in a natural way and work around the LLMs idiosyncrasies was important for achieving good results in the end.

**Vector search.** Had issues with InterSystems and getting the vector database to work.

## Accomplishments that we're proud of

**BAIB is significantly more powerful than we thought.** As we played around with BAIB and asked fun questions like “what are the weirdest texts that Tony has sent me?”, its in-depth analysis on Tony’s weird texts were incredibly accurate: “Tony mentions that maybe his taste buds have become too American… This reflection on cultural and dietary shifts is interesting and a bit unusual in the context of a casual conversation.” This has increased our conviction in the long-term potential of this idea. We truly believe that this product must and will exist with or without us.

**Our team organization was good (for a hackathon).** We split our team into the backend team and the frontend team. We’re proud that we made something useful and beautiful.

## What we learned

Prompt engineering is very important. As we progressed through the project, we were able to speed up the response significantly and increase the quality by just changing the way we framed our question.
ChatGPT 4.0 is more expensive than we thought.
Further conviction that personal assistants will have a huge stake in the future.
Energy drinks were not as effective as expected.

## What's next for BAIB

Building this powerful prototype gave us a glimpse of what BAIB could really become. We believe that BAIB can be integrated into all aspects of life. For example, integrating with other communication methods like Discord, Slack, and Facebook will allow the personal assistant to gain a level of organization and analysis that would not be previously possible.

Imagine getting competing offers at different companies and being able to ask BAIB, who can combine the knowledge of the internet with the context of your family and friends to help give you enough information to make a decision.

We want to continue the development of BAIB after this hackathon and build it as an app on your phone to truly become the Best AI Buddy.
## Inspiration

**The Immigration Story**

While brainstorming a problem to tackle in this challenge, we were intrigued by the lengths that many immigrants are willing to take for a new beginning. We wanted to come up with a way to try and ease this oftentimes difficult, intimidating transition to a new country and culture by creating a platform that would allow immigrants to connect with communities and resources relating to them. Current politics highlight the rising crisis of immigration and the major implications that it could have on current and future generations. An immigrants story does not end after they arrive in the US, their struggle can continue for years after their move. Hopefully, Pangea can bridge the gap between immigrants and their new environment, improving their lives in their new home.

## What is It?

**There are no borders in Pangea**

Pangea provides a user-friendly platform that immigrants can use to locate and connect with cultural resources, communities, and organizations near them. Our website fosters a close connection to cultural centers and allows immigrants to easily find resources near them that can ease their transition into the US. Some of Pangea's major features include an interactive heat map that shows various restaurants, shops, community centers, and resources based on the users' selected features and a plug-in to the telegram social media platform, an app commonly used by many immigrants. This plug-in links directly to a translation bot that allows users to practice their english or perform a quick translation if they need to do so.

## How We Built It

**Used HTML, Javascript, CSS, PHP, a Google Map API, NLP, and a Telegram plugin**

Using HTML and a Google Maps API we created a homepage and a heat map showing resources in the area that may be useful for immigrants. For the plug in, we found a Telegram plugin online that was simple to integrate into our code and create a helpful little translator accessible via a small bubble in the homepage.

We also researched data for a couple of ethnicities and created code that added these data points to the heat map to show densities of what resources are where.

## Challenges

As with any project, it was not always smooth sailing. Throughout our time creating this platform, we ran into many problems such as dealing with an unfamiliar language (most of us had not used Javascript before), tackling new programming techniques (such as building a website, creating a heat map, and utilizing API's and plug-ins), and fighting through the growing exhaustion as our lack of sleep caught up to us.

## Accomplishments

Our team is extremely proud of what we have accomplished this weekend. As a team composed of freshman and sophomores with little to no experience with hackathons, we were able to produce a working website that incorporates many different features. Despite not having done many of the things we attempted this weekend, we succeeded in using plug-ins, an API, and natural language processing in our prototype. We also worked very well together as a team and formed bonds that will last long after HackMIT ends.

## What We Learned

Throughout this experience, we learned many new skills that we will now be able to take with us in our future studies and projects. A few of the things we learned are listed below:

* How to build a website
* How to install a plug-in
* How to integrate API's into our website
* Natural Language Processing (NLP)
* How to create a heat map
* How to code in Javascript

## What's Next for Pangea?

**Pangea knows no bounds**

The applications for this site are near limitless. We hope to see Pangea grow as not only a resource for immigrants, but also a means of connection between them. In the future, we would like to incorporate profiles into our website, even expanding it into an app to create a social network for immigrants to connect with people of similar backgrounds, different cultures, and resources such as translators, lawyers, or social activists. In addition, we would like to add more data to our heat map to expand our reach past the local span of the Boston area. We plan to do this by scraping more data from the internet and including a more diverse scope of cultures in our database. Finally, we hope to further refine the translation bot by adding more language options so that Pangea will expand to even more cultural groups.
## Inspiration

Have you ever wondered what's actually in your shampoo or body wash? Have you ever been concerned about the toxicity of certain chemicals in them on your body and to the environment?

If you answered yes, you came to the right place. Welcome to the wonderful world of Goodgredients! 😀

Goodgredients provides a simple way to answer these questions. But how you may ask.

## What it does

Goodgredients provides a simple way to check the toxicity of certain chemicals in them on your body and to the environment. Simply take a picture of your Shampoo or body wash and check which ingredient might harmful to you.

## How I built it

The project built with React Native, Node JS, Express js, and Einstein API. The backend API has been deployed with Heroku.

The core of this application is Salesforce Einstein Vision. In particular, we are using Einstein OCR (Optical Character Recognition), which uses deep learning models to detect alphanumeric text in an image. You can find out more info about Einstein Vision here.

Essentially, we've created a backend api service that takes an image request from a client, uses the Einstein OCR model to extract text from the image, compares it to our dataset of chemical details (ex. toxicity, allergy, etc.), and sends a response containing the comparison results back to the client.

## Challenges I ran into

As first-time ReactNative developers, we have encountered a lot of environment set up issue, however, we could figure out within time!

## Accomplishments that I'm proud of

We had no experience with ReactNative but finished project with fully functional within 24hours.

## What I learned

## What's next for Goodgredients