## Inspiration

The inspiration came from working with integrated BMCs on server motherboards. These devices are able to control servers remotely without the need of an operating system on the server. This is accomplished by streaming the servers' VGA input and attaching a keyboard via USB. The major pain points about BMCs are that they are normally embedded on the servers, the streamed keyboard input is prone to errors and that since the screen is streamed the screen is not interactable. Our prescriptive design was a small portable device that has the same functionality as the BMC while solving these pain points and adding smart home functionality features.

## Challenges we ran into

Originally, we planned to use the Raspberry Pi 3 because it had an onboard network module which eliminates one step in the integration process. Unfortunately, the Raspberry Pi 3 cannot emulate USB HID devices such as a keyboard, which made it infeasible to use. Once we switched to the Raspberry Pi Pico, which could emulate a keyboard but didn’t have WIFi, we had to integrate it with a network module ESP8266 or ESP32. We encountered difficulties powering the module through the RPI and later discovered that both require 5v while our RPI only provided 3.3v. Furthermore, the RPI Pico does not have USB ports to support the display capture feature we aimed to implement, instead we added additional features.

## What it does

Marcomancer main features are the following:

* Run Macros Remotely via USB Keyboard Emulation
* Programmable Smart Home Voice Activated Macros
* Robust Input Streaming via Web App
* Portable Plug and Play

These features solve two of the major BMC pain points of no portability and error prone input. The system also allows system administrators to reliably program macros to remote servers. Macromancer is not only for tech savvy individuals but also provides smart accessibility to anyone through voice control on their phones to communicate to their PC. As long as you can accomplish a task with just keyboard inputs, you can use Macromancer to send a macro from anywhere.

## How we built it

Hardware: The RPI Pico that waits for keycodes transmitted by the ESP32. ESP32 polls our cloud server for commands.

Software: Google Assistant sends predefined user commands to the cloud server webhook via IFTTT applets. Users can also use our Frontend Web App to manually queue command strings and macros for the ESP32 to poll via a GET API.

## Accomplishments that we're proud of

* Ability for users to add/delete any new commands
* Integrating with the network modules
* The minimum latency

## What we learned

* Flashing firmware on microprocessors
* Emulating keyboard through microprocessors (Adafruit)
* TCP and Checksum error detection
* Activating webhooks with Google Assistant
* How to communicate between microprocessors

## What's next for Macromancer

* Better security (encryption)
* RPI 4 (reduce hardware and allow display capture for text editing)
* Wake on LAN
VIDEO LINK: <https://drive.google.com/file/d/1eBhTndCY_TOc1Exnojuell04R3JYgWQA/view?usp=sharing>

## Inspiration

Ever been a panelist at a virtual event speaking into the dark abyss of cameras off?

What about running a virtual event? Setting up your Google Slide, a Slido for audience Q&A, a Spotify playlist to share during breaks, and a feedback form to share with all participants... only to realize you forgot to open one of those tabs and you end up screen sharing your entire desktop with your spicy dms to your significant other while trying to find your other tab.

These are pain points we face every day as execs on student clubs like the Women in Tech Society, where we plan virtual events for 40-70 students every month. For our virtual events, we consistently have to screen-share 3-4 different platforms for a successful event. We also have to get creative with how we engage with event attendees, which just adds more frustration to our event planning.

We needed a solution that could:

1. Combine the features of our favourite event planning platforms to increase event planning efficiency
2. Creatively increase engagement with event attendees who have cameras off

We call this "Fusion"

## What it does

Fusion is an all in one event planning platform, that combines features of our favourite event planning platforms into one simple webpage that can be screen shared during an online event.
For event moderators, Fusion is a webpage that:

* Integrates with your Google Slides presentation
* Integrates with Spotify for welcome & break music
* Integrates with Slido for easy audience Q&A
* Send out and pin important announcements, which normally get lost in chats
* Has the ability to email all participants after an event (thank you notes and feedback forms)

For event attendees, Fusion is a simple webpage that:

* Allows them to pick a customized zoom background for the event to encourage cameras on
* Integrates with Slido for easy Q&A without visiting another platform
* Allows them to see event details, speakers & announcements

Fusion is made to be simple. Event attendees don’t have to register or create an account, and can access Fusion in all its glory through the event specific URL, which is easy to share.

## How we built it

Tech Stack: MERN (Mongodb, Express.js, React.js, Node.js), Twilio Sendgrid for automated emails
Design: Figma

* Event settings are sent in through the Admin Portal (client side) and uploaded to our database by accessing the Node backend endpoints using Express (server)
* Database is queried for a specific event ID and the appropriate content is displayed for admins and attendees
* Attendees join an event using the unique event ID and their info is added to our database
* With the click of a button, the Twilio Sendgrid integration sends a customized email to all attendees

Check it out in action here!
<https://drive.google.com/file/d/1c_XGoqwbMdeX1HgR-qO9FQRSfVnyyhMD/view?usp=sharing>

## Challenges we ran into

Elaine: This is my first time working with Node and Express and had challenges getting accustomed to JavaScript. (If you see pythonic-style JavaScript, it was probably committed by me!) Because of this learning curve, I had difficulties making requests to our backend using Axios. It was also my first time setting up a Mongodb database so there was a bit of learning here as well.

Eng: I think the most challenging part about this Hackathon was collaborating in a team setting and learning to resolve merge conflicts with Git!

## Accomplishments that we're proud of

As event organizers ourselves, we’re so proud of creating a tool that will help us solve many of our pain points running events in the future. We hope to continue to work on Fusion so that we can launch it and use Fusion in some of our own events!

## What we learned

For some of us, we learned a lot about new languages and tools such as Node.js, Mongodb, and Twilio. It was also fun to learn more about the virtual event space during our preliminary research and better understand some of the most common pain points experienced by both attendees and organizers. We were also able to look into other tools and APIs and get a good sense of next steps we want to take with this app!

## What's next for Fusion

We hope to continue work on this after the hackathon and hope to turn into a mini Startup. We are hoping to integrate with the Zoom API to leverage some of its features such as chat and event creation abilities.
## Inspiration

One day Saaz was sitting at home thinking about his fitness goals and his diet. Looking in his fridge, he realized that, on days when his fridge was only filled with leftovers and leftover ingredients, it was very difficult for him to figure out what he could make that followed his nutrition goals. This dilemma is something Saaz and others like him often encounter, and so we created SmartPalate to solve it.

## What it does

SmartPalate uses AI to scan your fridge and pantry for all the ingredients you have at your disposal. It then comes up with multiple recipes that you can make with those ingredients. Not only can the user view step-by-step instructions on how to make these recipes, but also, by adjusting the nutrition information of the recipe using sliders, SmartPalate caters the recipe to the user's fitness goals without compromising the overall taste of the food.

## How we built it

The scanning and categorization of different food items in the fridge and pantry is done using YOLOv5, a single-shot detection convolutional neural network. These food items are sent as a list of ingredients into the Spoonacular API, which matches the ingredients to recipes that contain them. We then used a modified natural language processing model to split the recipe into 4 distinct parts: the meats, the carbs, the flavoring, and the vegetables. Once the recipe is split, we use the same NLP model to categorize our ingredients into whichever part they are used in, as well as to give us a rough estimate on the amount of ingredients used in 1 serving. Then, using the Spoonacular API and the estimated amount of ingredients used in 1 serving, we calculate the nutrition information for 1 serving of each part. Because the amount of each part can be increased or decreased without compromising the taste of the overall recipe, we are then able to use a Bayesian optimization algorithm to quickly adjust the number of servings of each part (and the overall nutrition of the meal) to meet the user's nutritional demands. User interaction with the backend is done with a cleanly built front end made with a React TypeScript stack through Flask.

## Challenges we ran into

One of the biggest challenges was identifying the subgroups in every meal(the meats, the vegetables, the carbs, and the seasonings/sauces). After trying multiple methods such as clustering, we settled on an approach that uses a state-of-the-art natural language model to identify the groups.

## Accomplishments that we're proud of

We are proud of the fact that you can scan your fridge with your phone instead of typing in individual items, allowing for a much easier user experience. Additionally, we are proud of the algorithm that we created to help users adjust the nutrition levels of their meals without compromising the overall taste of the meals.

## What we learned

Using our NLP model taught us just how unstable NLP is, and it showed us the importance of good prompt engineering. We also learned a great deal from our struggle to integrate the different parts of our project together, which required a lot of communication and careful code design.

## What's next for SmartPalate

We plan to allow users to rate and review the different recipes that they create. Additionally, we plan to add a social component to SmartPalate that allows people to share the nutritionally customized recipes that they created.