## Inspiration

Inspired by products such as Tab and Humane AI Pin, we were driven to create a conversational AI similar to C3PO from Star Wars.

## What it does

A portable bot in a comforting form factor that you can conversate with on the fly, and it always at the ready.

## How we built it

We built an API with speech-to-text and text-to-speech algorithms that send audio input from the user, turn it into text, and send it to the OpenAI API. The response from OpenAI is then converted into an audio file and sent back to the client. We used a raspbery pi as a client that enables portability of the product, and uses a button on a breadboard to record audio and call the API we created.

## Challenges we ran into

Making API calls in python and sending and receiving .wav files to the API was significantly challenging.

## Accomplishments that we're proud of

building a functional API server with TTS and STT capabilities using azure and openai, and building with hardware for the first time, being able to record audio at the touch of a button.

## What we learned

How to use raspberry pis and use it as a client to call an API

## What's next for BrodieAI

Vision capabilities, enabling it to answer queries about its surroundings after gathering that context and processing it with a computer vision model. (Model already created, need to integrate it into API and enable video/photo captures to be sent to our API)
## Inspiration

Reading the news helps people expand their knowledge and broaden their horizons. However, it can be time-consuming and troublesome to find quality news articles and read lengthy, boring chunks of text. Our goal is to make news **accessible** to everyone. We provide **concise**, **digestible** news **summaries** in a **conversational** manner to make it as easy as possible for anyone to educate themselves by reading the news.

## What it does

News.ai provides a concise and digestible summary of a quality article related to the topic you care about. You can easily ask **follow-up questions** to learn more information from the article or learn about any related concepts mentioned in the article.

## How we built it

1. We used *React.js* and *Flask* for our web app.
2. We used *NewsAPI* to recommend the most updated news based on preferences.
3. We used *Monster API's OpenAI-Whisper API* for speech-to-text transcription.
4. We used *Monster API's SunoAI Bark API* for text-to-speech generation.
5. We used *OpenAI's GPT 4 API* large language model (LLM) to provide summaries of news articles.

## Challenges we ran into

We ran into the challenge of connecting multiple parts of the project. Because of its inherent complexity and interconnectivity, making different APIs and frontend plus backend to work together has been our most difficult task.

## Accomplishments that we're proud of

We're happy that we established a strong pipeline of API calls using AI models. For example, we converted the user's audio input to text using Whisper API before generating text in response to the user's request using GPT API and finally, we converted the generated text to audio output using Bark API. We are also proud to have integrated the NewsAPI in our recommendation system so we can display the latest news for each user tailored to their preferences.

## What we learned

Each of our team members had a deep understanding of a specific part of our tech stack; whether that be the frontend, backend, or usage of AI/LLM models and APIs. We learned a lot about how these tools can be integrated and applied to solve real-world problems.

Furthermore, by spending the first day going booth to booth and speaking individually to every sponsor, we learned about the intricacies of each platform and API. This allowed us to build a platform that synthesized the strengths of various tools and technologies. For example, we were able to take advantage of the ease and scalability of Monster API's Whisper and Bark APIs.

## What's next for News.ai

Moving forward, we hope to allow for more personalized search of news articles beyond generic topics. Furthermore, we hope to collect additional personalized characteristics that improve the podcast content and understanding for users.
## Inspiration

Partially inspired by the Smart Cities track, we wanted our app to have the direct utility of ordering food, while still being fun to interact with. We aimed to combine convenience with entertainment, making the experience more enjoyable than your typical drive-through order.

## What it does

You interact using only your voice. The app automatically detects when you start and stop talking, uses AI to transcribe what you say, figures out the food items (with modifications) you want to order, and adds them to your current order. It even handles details like size and flavor preferences. The AI then generates text-to-speech audio, which is played back to confirm your order in a humorous, engaging way. There is absolutely zero set-up or management necessary, as the program will completely ignore all background noises and conversation. Even then, it will still take your order with staggering precision.

## How we built it

The frontend of the app is built with React and TypeScript, while the backend uses Flask and Python. We containerized the app using Docker and deployed it using Defang. The design of the menu is also done in Canva with a dash of Harvard colors.

## Challenges we ran into

One major challenge was getting the different parts of the app—frontend, backend, and AI—to communicate effectively. From media file conversions to AI prompt engineering, we worked through each of the problems together. We struggled particularly with maintaining smooth communication once the app was deployed. Additionally, fine-tuning the AI to accurately extract order information from voice inputs while keeping the interaction natural was a big hurdle.

## Accomplishments that we're proud of

We're proud of building a fully functioning product that successfully integrates all the features we envisioned. We also managed to deploy the app, which was a huge achievement given the complexity of the project. Completing our initial feature set within the hackathon timeframe was a key success for us. Trying to work with Python data type was difficult to manage, and we were proud to navigate around that. We are also extremely proud to meet a bunch of new people and tackle new challenges that we were not previously comfortable with.

## What we learned

We honed our skills in React, TypeScript, Flask, and Python, especially in how to make these technologies work together. We also learned how to containerize and deploy applications using Docker and Docker Compose, as well as how to use Defang for cloud deployment.

## What's next for Harvard Burger

Moving forward, we want to add a business-facing interface, where restaurant staff would be able to view and fulfill customer orders. There will also be individual kiosk devices to handle order inputs. These features would allow *Harvard Burger* to move from a demo to a fully functional app that restaurants could actually use. Lastly, we can sell the product by designing marketing strategies for fast food chains.