## Inspiration

We're computer science students, need we say more?

## What it does

"Single or Nah" takes in the name of a friend and predicts if they are in a relationship, saving you much time (and face) in asking around. We pull relevant Instagram data including posts, captions, and comments to drive our Azure-powered analysis. Posts are analyzed for genders, ages, emotions, and smiles -- with each aspect contributing to the final score. Captions and comments are analyzed for their sentiment, which give insights into one's relationship status. Our final product is a hosted web-app that takes in a friend's Instagram handle and generate a percentage denoting how likely they are to be in a relationship.

## How we built it

Our first problem was obtaining Instagram data. The tool we use is a significantly improved version of an open-source Instagram scraper API (<https://github.com/rarcega/instagram-scraper>). The tool originally ran as a Python command line argument, which was impractical to use in a WebApp. We modernized the tool, giving us increased flexibility and allowing us to use it within a Python application.

We run Microsoft's Face-API on the target friend's profile picture to guess their gender and age -- this will be the age range we are interested in. Then, we run through their most recent posts, using Face-API to capture genders, ages, emotions, and smiles of people in those posts to finally derive a sub-score that will factor into the final result. We guess that the more happy and more pictures with the opposite gender, you'd be less likely to be single!

We take a similar approach to captions and comments. First, we used Google's Word2vec to generate semantically similar words to certain keywords (love, boyfriend, girlfriend, relationship, etc.) as well as assign weights to those words. Furthermore, we included Emojis (is usually a good giveaway!) into our weighting scheme[link](https://gist.github.com/chrisfischer/144191eae03e64dc9494a2967241673a). We use Microsoft's Text Analytics API on this keywords-weight scheme to obtain a sentiment sub-score and a keyword sub-score.

Once we have these sub-scores, we aggregate them into a final percentage, denoting how likely your friend is single. It was time to take it live. We integrated all the individual calculations and aggregations into a Django., then hosted all necessary computation using Azure WebApps. Finally, we designed a simple interface to allow inputs as well as to display results with a combination of HTML, CSS, JavaScript, and JQuery.

## Challenges we ran into

The main challenge was that we were limited by our resources. We only had access to basic accounts for some of the software we used, so we had to be careful how on often and how intensely we used tools to prevent exhausting our subscriptions. For example, we limited the number of posts we analyzed per person. Also, our Azure server uses the most basic service, meaning it does not have enough computing power to host more than a few clients.

The application only works on "public" Instagram ideas, so we were unable to find a good number of test subjects to fine tune our process. For the accounts we did have access to, the application produced a reasonable answer, leading us to believe that the app is a good predictor.

## Accomplishments that we're proud of

We proud that we were able to build this WebApp using tools and APIs that we haven't used before. In the end, our project worked reasonably well and accurately. We were able to try it on people and get a score which is an accomplishment in that. Finally, we're proud that we were able to create a relevant tool in today's age of social media -- I mean I know I would use this app to narrow down who to DM.

## What we learned

We learned about the Microsoft Azure API (Face API, Text Analytics API, and web hosting), NLP techniques, and full stack web development. We also learned a lot of useful software development techniques such as how to better use git to handle problems, creating virtual environments, as well as setting milestones to meet.

## What's next for Single or Nah

The next steps for Single or Nah is to make the website and computations more scalable. More scalability allows more people to use our product to find who they should DM -- and who doesn't want that?? We also want to work on accuracy, either by adjusting weights given more data to learn from or by using full-fledged Machine Learning. Hopefully more accuracy would save "Single or Nah" from some awkward moments... like asking someone out... who isn't single...
## Inspiration

There's something about brief glints in the past that just stop you in your tracks: you dip down, pick up an old DVD of a movie while you're packing, and you're suddenly brought back to the innocent and carefree joy of when you were a kid. It's like comfort food.

So why not leverage this to make money? The ethos of nostalgic elements from everyone's favourite childhood relics turns heads. Nostalgic feelings have been repeatedly found in studies to increase consumer willingness to spend money, boosting brand exposure, conversion, and profit.

## What it does

Large Language Marketing (LLM) is a SaaS built for businesses looking to revamp their digital presence through "throwback"-themed product advertisements.

Tinder x Mean Girls? The Barbie Movie? Adobe x Bob Ross? Apple x Sesame Street? That could be your brand, too. Here's how:

1. You input a product description and target demographic to begin a profile
2. LLM uses the data with the Co:here API to generate a throwback theme and corresponding image descriptions of marketing posts
3. OpenAI prompt engineering generates a more detailed image generation prompt featuring motifs and composition elements
4. DALL-E 3 is fed the finalized image generation prompt and marketing campaign to generate a series of visual social media advertisements
5. The Co:here API generates captions for each advertisement
6. You're taken to a simplistic interface where you can directly view, edit, generate new components for, and publish each social media post, all in one!
7. You publish directly to your business's social media accounts to kick off a new campaign 🥳

## How we built it

* **Frontend**: React, TypeScript, Vite
* **Backend**: Python, Flask, PostgreSQL
* **APIs/services**: OpenAI, DALL-E 3, Co:here, Instagram Graph API
* **Design**: Figma

## Challenges we ran into

* **Prompt engineering**: tuning prompts to get our desired outputs was very, very difficult, where fixing one issue would open up another in a fine game of balance to maximize utility
* **CORS hell**: needing to serve externally-sourced images back and forth between frontend and backend meant fighting a battle with the browser -- we ended up writing a proxy
* **API integration**: with a lot of technologies being incorporated over our frontend, backend, database, data pipeline, and AI services, massive overhead was introduced into getting everything set up and running on everyone's devices -- npm versions, virtual environments, PostgreSQL, the Instagram Graph API (*especially*)...
* **Rate-limiting**: the number of calls we wanted to make versus the number of calls we were allowed was a small tragedy

## Accomplishments that we're proud of

We're really, really proud of integrating a lot of different technologies together in a fully functioning, cohesive manner! This project involved a genuinely technology-rich stack that allowed each one of us to pick up entirely new skills in web app development.

## What we learned

Our team was uniquely well-balanced in that every one of us ended up being able to partake in everything, especially things we hadn't done before, including:

1. DALL-E
2. OpenAI API
3. Co:here API
4. Integrating AI data pipelines into a web app
5. Using PostgreSQL with Flask
6. For our non-frontend-enthusiasts, atomic design and state-heavy UI creation :)
7. Auth0

## What's next for Large Language Marketing

* Optimizing the runtime of image/prompt generation
* Text-to-video output
* Abstraction allowing any user log in to make Instagram Posts
* More social media integration (YouTube, LinkedIn, Twitter, and WeChat support)
* AI-generated timelines for long-lasting campaigns
* AI-based partnership/collaboration suggestions and contact-finding
* UX revamp for collaboration
* Option to add original content alongside AI-generated content in our interface
## Inspiration

It all started on Galentine's Day when our team found ourselves in a frenzy trying to track down a cake recipe we had seen on Instagram. Despite our best efforts, we couldn't locate the post amidst the sea of saved content across platforms like TikTok, Instagram, and Twitter. This experience led us to a realization: the lack of a robust search mechanism for managing our social media bookmarks was a widespread problem.

Bookmarks.ai revolutionizes the way you organize and search through your saved bookmarks across social media platforms.

## What it does

Bookmarks.ai serves as a platform to aggregate your saved posts (bookmarks) throughout your social media platforms and then runs a semantic search on the captions and hashtags of the post to retrieve relevant posts to your query when you are searching for specific content. In addition, it leverages k-means clustering and sentence similarity to group similar posts together in clusters, which are available on the home page for easy access and browsing.

## How we built it

1. We began by customizing the Instagram API which collects post metadata such as the image link, the web link, the caption, and the post ID. From there, we stored this information in the format of a JSON file.
2. These captions were then split using a CharacterTextSplitter, embedded with Langchain, and the vector embeddings were stored using MongoDB Atlas.
3. After creating a vector search index in MongoDB Atlas, we were able to successfully run queries with semantic search and the post details of the relevant posts would be returned as the result.
4. Based on the array of post captions, we were able to run K-Means Clustering using sentence similarity based on a model we found on HuggingFace to divide the posts into clusters. We were able to leverage prompt engineering and an OpenAI LLM to automatically come up with names/descriptions of each cluster (such as "cooking" or "fashion").
5. Our front end was built using React.js, HTML, and CSS

Technologies and Frameworks: React.js, HTML, CSS, Python, LangChain, HuggingFace, MongoDB, OpenAI

## Challenges we ran into

One of the biggest challenges we ran into was Instagram authentication and being able to scrape Instagram-saved posts. Since most social media apps have strict authentication guidelines, accessing embedding URLs for each Instagram post was often difficult. There was also a rate limit to Instagram's API. Additionally, all of our teammates were new to semantic search and k-means clustering so it took us a while to figure out how to create vector embeddings and the vector search index. It was also difficult to integrate the backend with the frontend as our backend would often return JSON files consisting of web links to posts, and we faced difficulty converting them to embedding codes as the Meta Developer Tools API that was used to do that was often unpredictable in generating access tokens.

## Accomplishments that we're proud of

This was our first time exploring semantic search and k-means clustering. We were excited that we were able to build a project with both of these features fully working by the end of the project. We were also excited about the problem we were solving—we had personally faced challenges managing our bookmarks and it was cool to use LLMs in a meaningful way to make that process significantly easier.

## What we learned

This was a great opportunity for all of us to learn about new frameworks and technologies we hadn't used before, become comfortable debugging them, and quickly come up with a backup plan when things weren't going smoothly in the end. Ultimately, we were able to create a product that we believe would be personally useful to us.

## What's next for Bookmarks.ai

We would love to include AI agents that allow you to automatically create action items from posts that you save and query (think creating a grocery list if you view a baking post). We would also love to be able to intelligently process the images in posts and search based on that as sometimes captions and hashtags may not be enough information.