[Example brainrot output](https://youtube.com/shorts/vmTmjiyBTBU)

[Demo](https://youtu.be/W5LNiKc7FB4)

## Inspiration

Graphic design is a skill like any other, that if honed, allows us to communicate and express ourselves in marvellous ways. To do so, it's massively helpful to receive specific feedback on your designs. Thanks to recent advances in multi-modal models, such as GPT-4o, even computers can provide meaningful design feedback.

What if we put a sassy spin on it?

## What it does

Adobe Brainrot is an unofficial add-on for Adobe Express that analyzes your design, creates a meme making fun of it, and generates a TikTok-subway-surfers-brainrot-style video with a Gordon Ramsey-esque personality roasting your design. (Watch the attached video for an example!)

## How we built it

The core of this app is an add-on for Adobe Express. It talks to a server (which we operate locally) that handles AI, meme-generation, and video-generation.

Here's a deeper breakdown:

1. The add-on screenshots the Adobe Express design and passes it to a custom-prompted session of GPT-4o using the ChatGPT API. It then receives the top design issue & location of it (if applicable).
2. It picks a random meme format, and asks ChatGPT for the top & bottom text of said meme in relation to the design flaw (e.g. "Too many colours"). Using the memegen.link API, it then generates the meme on-the-fly and insert it into the add-on UI.
3. Using yt-dlp, it downloads a "brainrot" background clip (e.g. Subway Surfers gameplay). It then generates a ~30-second roast using ChatGPT based on the design flaw & creates a voiceover using it, using OpenAI Text-to-Speech. Finally, it uses FFmpeg to overlay the user's design on top of the "brainrot" clip, add the voiceover in the background, and output a video file to the user's computer.

## Challenges we ran into

We were fairly unfamiliar with the Adobe Express SDK, so it was a learning curve getting the hang of it! It was especially hard due to having two SDKs (UI & Sandbox). Thankfully, it makes use of existing standards like JSX.

In addition, we researched prompt-engineering techniques to ensure that our ChatGPT API calls would return responses in expected formats, to avoid unexpected failure.

There were quite a few challenges generating the video. We referenced a project that did something similar, but we had to rewrite most of it to get it working. We had to use a different yt-dl core due to extraction issues. FFmpeg would often fail even with no changes to the code or parameters.

## Accomplishments that we're proud of

* It generates brainrot (for better or worse) videos
* Getting FFmpeg to work (mostly)
* The AI outputs feedback
* Working out the SDK

## What we learned

* FFmpeg is very fickle

## What's next for Adobe Brainrot

We'd like to flesh out the UI further so that it more *proactively* provides design feedback, to become a genuinely helpful (and humorous) buddy during the design process. In addition, adding subtitles matching the text-to-speech would perfect the video.
## Inspiration

All our team members enjoy reading as a hobby across numerous genres. Unfortunately, reading comprehension and literacy overall have been declining, which poses a massive problem for future generations. There are countless factors at play here, and we can't solve all of them, but we can definitely work to kindle a spark that might just blossom into a love for reading. Many of these adjacent tools are made for classroom settings. On the contrary, we're built for students, by students, and by readers, for readers.

## What it does

Storyscape is an application aimed at developing elementary and middle schoolers' grasp over the English language. Turn on your mic and get ready to read some words! To be specific, we have identified three core skills to be problematic: diction, sentence structure variety, and expressiveness. You will be given an unlimited amount of examples for each of these and the opportunity to practice and replicate these examples. With the help of our cat assistant, you can learn how to read passages expressively and eloquently!

## How we built it

Storyscape is built on five major technologies:

1. Google Gemini Flash -- speedy, cost-efficient, great at storytelling
2. Hume AI -- personable, easy to integrate TTS, and great for sentiment analysis
3. React/Next.js -- a popular JS framework that simplifies many best practices built for vercel
4. Vercel -- easy deployment to the web
5. Firebase -- a handy tool for auth, functions, storage and analytics

## Challenges we ran into

One major problem we faced during the competition was the cold weather at night. We struggled to get some quality sleep due to the lack of comfortable sleeping arrangements :(

## Accomplishments that we're proud of

* We would actually use this app ourselves after some minor improvements!
* Created a polished and aesthetically pleasing web app demonstrating AI technologies
* Stayed up 24 hours in a row

## What we learned

This hackathon was a great opportunity to learn about all the different AI technologies that are out there and how we can implement them into our lives. We learned about sentiment analysis, some handy prompt engineering, and some techniques to interface with AI in a smart way.

## What's next for Storyscape

We're still working on adding themes into the text adventures. With better prompt engineering and utilisation of the best models to foster a closer connection to the text, Storyscape can become a true companion for those who want to augment their English skills.
![Glyph](https://i.ibb.co/NZybN1z/Untitled-removebg-preview.png)

## Inspiration

Our team has been users of Adobe Express for a while, using it to create designs for schoolwork, extracurriculars, and often personal designs such as music covers. Seeing the opportunity to make a plugin as a challenge piqued our interests, and we thought it'd be a suitable project to introduce creative features that leverage artificial intelligence to improve our own user experience.

## What it does

Glyph is a suite of plugins that will improve your designs & marketing experience. It comes with the following suite of products:

* **moshfilters**: A group of filters that create unique filter type of designs, making use of technologies such as data moshing, vaporwave, and microwave.
* **captionate**: A plugin capable of downloading your current design and generating captions with keywords specified by the user to maximize social media attention.
* **ai sticker**: A plugin intended to use GPT 4's newly released vision API to have the ability to insert stickers with ease whenever the user intends, further maximizing ability for design creativity.

## How we built it

Our suite of plugins are built with modern and scalable technologies. We made use of Adobe Express's add-on CLI to start with the base of our add-on, made use of React to create our frontend components, along with other frameworks such as MaterialUI. Our backend is largely composed of Adobe's Document Sandbox API as well as their Communication API, which allows our frontend to directly interact with the Express's document. It also makes use of Axios as our promise based HTTP request library.

## Challenges we ran into

Struggled getting onboarded with the Express Communication API, and figuring out how the server architecture would look, specifically with the placement & location of our calls -- we intended to maintain clean code, however, at times it was a challenge. Adobe Express's documentation was largely helpful throughout.

Our suite of plugins often make external API calls to different artificial intelligence models, such as OpenAI's GPT4; however, cross origin policy issues would often arise as we're making calls from an Adobe hostname to external APIs.

## Accomplishments that we're proud of

* Implementing arithmetic to perform pixel manipulation to create unique filters.
* User Interface making use of standard practices with respect to UX principles and accessible design.
* Leveraging artificial intelligence APIs, such as GPT4 Vision, and Segmind VegaRT to create unique functionality within Adobe Express. 😁

## What we learned

A lot-- the Adobe Sandbox functionality, along with the communication API, new React features, and handling cross origin requests on a plugin!

## What's next for glyph

Publishing our plugin to the vast array of existing Adobe Express marketplace, along with UI polishing.