## 💡 Inspiration 💡

>
> *“There is no lack of educative content, but the correct delivery”*
>
>
>

In an age where educational content is abundant, the challenge lies not in its availability but in its effective delivery and the unique perspective of the presenter. While technology like Google Search, ChatGPT, and advanced AI search bots like Perplexity.ai have made accessing content effortless, education efficiency isn’t limited by the speed of information queries. It is, rather, bounded by the speed of your brain’s information intake, which can only be boosted via proactively engaging, personalized, and even addictive learning experiences.

We planned to achieve this via 5 tools:

1. 🖼Multimedia: LLM, text-to-image, AI Agent, and web-scraping enable a textual, visual, and interactive experience
2. 🔎Diverse Content Perspectives: By tweaking prompts with different keywords, we can simulate a variety of content creators, offering a rich spectrum of views and styles.
3. 📈Recommendation Algorithms: With LLMs, tagging and describing content becomes scalable and cost-effective, enhancing the efficiency of recommendation systems.
4. 🎮Gamification: Use gamification to improve engagement.
5. 💰Financial incentive: LLM automatically turns learners (question-askers and answer-seekers) into teachers (answer-generators) by reusing answers. In short, people get compensated for asking valuable frequently asked questions.

These elements have proved to be crucial for any content platform seeking viral success. Fortunately, the advent of Large Language Models (LLMs) has revolutionized the potential of content platforms, making it easier than ever to satisfy these critical aspects.

## 💻 What it does 💻

### 1️⃣Recommendation Driven Generation

We track and collect the user’s behavior data and interaction with the feed, and gradually learn the user's interest profile. Then, the learned profile, represented by a set of weighted tags, is used to accurately and automatically generate the user’s favoured content.

### 2️⃣Learning Journey Decomposer

Unlike ChatGPT or Perplexity, which spit out ineffective, lengthy responses when given users’ complex learning goals, we decompose them into a sequence of general and reusable learnable modules. This sequence, what we call a “learning journey”, gives just the needed amount of knowledge of every necessary aspect for you to reach your learning goal. Furthermore, since each module is reusable, it not only makes your learning transferable and generalizable but also boosts other user’s usage of your generated content, thereby boosting your reward.

### 3️⃣Subjectiveness, but safely and beneficially

Every content we generate is a RAG (Retrieval Augmented Generation) from trustworthy human-generated content on the Internet (with sources attached). This introduces subjectiveness, human nature, and opinions into the content, which is not achievable solely via the usage of any generative LLMs. While enforcing safety via negative prompts, the introduced subjectiveness brings humanized flavour to learners, hence making the content less rigid and more entertaining.

### 4️⃣ Gamified Mini Course Maker

Given a topic card, a gamified mini-course, with beautiful visuals and short multiple-choice questions, can be automatically created. Your performance will impact how much XP you will receive, which will impact your levelling progress.

## 👨‍💻How we built it 👨‍💻

How do we create contextual content for complex learning goals while reusing content?

### Content Generation 1️⃣- User Generated Content

Users can ask questions and are assisted by LLMs to tailor the learning goal into desirable modules for the user to interact with via natural language. The user can iterate on the division of content in modules and when satisfied, a similarity search using InterSystems is used to find existing content modules that can be reused. Existing modules are combined with newly generated modules to create a full “Learning Journey” that meets the user’s learning goals.

![User generated content flow](https://github.com/Treehacksiearn/iearn-diagrams/blob/main/User%20generated%20content.png?raw=true)

### Content Generation 2️⃣ - Automatically Generated Content

As the user scrolls their feed, we collect various data points like their CTR and interactions, which is used to assign weighted tags to the user interest profile. Users are recommended content modules based on their past activity using a custom recommendation algorithm on Pinecone. This is existing content that has been generated by other users that we think this user will find helpful using a combination of the user’s tags, and content tags, as well as tracking the user’s time spent and click-through rate on content.

However, because the content is customized specifically to the user’s interests and past questions, there could be a lack of “relevant" content for the user. Rather than showing the user something irrelevant, we utilized RSS feeds from forums like Reddit, Medium, and Quora to generate new “hot” questions that are specific to the user’s needs and interests. These questions follow the same flow described above to generate new cards, modules, and tags.

![Auto generated content flow](https://github.com/Treehacksiearn/iearn-diagrams/blob/main/Auto%20generated%20content.png?raw=true)

### Rest of the App

The application was built using React-Native and Expo for the front end and Flask and Supabase for the back end.

## 😬 Challenges we ran into 😬

#### HCI (Human Computer Interaction) - ✅solved

In the course-making page, it is hard to build an AI Agent that uses natural language to communicate and manipulate a designed Learning Journey JSON object. Solely using chat completion doesn’t work since actions, like add/delete/modify, are not guaranteed to be precise (can cause unexpected errors). We used the OpenAI assistant API with function calling to ensure stability, but the high usage cost held us back.

#### Scalable RecSys - ✅solved

While we have 2000+ generated learning cards, how do we build RecSys with a deep learning effect but no deep learning infra? Also, we don’t have enough data for the training.

#### User Behaviour Tracking and RecSys data collection - ✅solved

Content displayed in full screen doesn’t have CTR data. But unlike TikTok, where video playback completion rate can be easily collected, reading content’s completion rate requires tracking and making inferences on the user's reading speed. We utilized a Bayesian Inference as a fast and robust model to make inferences.

#### Not enough Pinecone indices - ❌unsolved

To determine reusable content, we need to compare the cosine similarity of card contents. But as a free tier, we only have 1 available index to use. Thus, reusability is unsolved due to a lack of resources. We pivoted to Intersystem for a local DB solution.

## 😤 Accomplishments that we're proud of 😤

* A complete Recommendation System: we built the entire user behaviour tracking, data collection, model, efficient candidate picking and ranking pipeline that achieves similar performance as DNN, but didn’t use any neural network
* Bring humanity and subjectiveness into AIGC (AI-generated content) - Through tags from learned user interest profiles, prompt engineering, web scraping, and querying popular websites’ RSS top feeding pages, we’re able to systematically determine what topic is viral, trending, and worthwhile generating, and generate human-like content through RAGs, with backed sources.
* Coercing GPT into returning JSON data in the new JSON mode was difficult and required a lot of prompt engineering to ensure that the output was stable and consistent

## 🧠 What we learned 🧠

* It is possible to systematically generate viral and enticing textual content.
Recommendation systems for full-screen content and non-full-screen content can be very different. Negative samples are hard to collect.
* There is a spectrum between casual reading casual learning and actual learning. LLM can create a smooth transition from the most casual to the most serious by developing knowledge details via creating stratified content and by adjusting styles from entertaining to profound.
Assistant AI is in beta because it doesn’t support JSON mode. This made the API much less useful and the AI agent swarm is yet to come.
* In the age of LLM, a single man can build a recommendation system, with no shortage of computing power and data.
* We had a lot of ambitious goals for Iearn as a product and it was important to focus on delivering an MVP before focusing on additional “nice-to-haves”.

## 🔭 What's next for Iearn? 🔭

* Polishing Assistant API AI Agent for the learning journey decomposer
* Build a functional API to keep track of nitty-gritty features for gamification
* Multiple card types and more multimedia: introduce music, video, cooking recipes, mini javascript games, etc… We have one type of content module but the possibilities are endless in the different formats of AI-generated media.
* Socialization, build social media features for more user interaction
## Inspiration

Current massive online courses are great resources to bridge the gap in educational inequality. Nevertheless, studies show that students often feel frustrated and have a high chance of giving up on their courses if there’s no support. We need to provide extra help to students to empower their potential through personalizing their learning experience.

## What it does

We are building an AI chatbot to enhance online learning. It will offer real-time support during online lectures (like MIT OCW) through interactive quizzes and personalized questions, promoting engagement and understanding, and helping to bridge the educational inequality gap.

Below are some of our products functions—

Answer Questions: Our platform can be used as a personal tutor. While having any questions about the video, pause and ask. Our bot will answer it for you, with simple language, outside resources, reference of this specific course (along which part of the video you should rewatch and what part of the text you should check out).

Summarization: For the long videos (particularly for those without sections), we will use a clustering algorithm along with LLMs to generate a summarization and sections for the video.

Pop up quiz: We will use our bot, fed with quiz and assignment from course website, to generate a series pop up quiz while you watching the video. This will first make sure that you are paying attention on the video and second that you are understanding the content.

Personalized active learning helper: We will also implement a function that allows you to select your current level of knowledge, vocabulary, and ability to understand language. By doing so, we allowed the bot to actively pop up tips and explanation of video content.

## How we built it

We used React as front end and Flask as back end. For now we will use ChatGPT as our AI engine.

## Challenges we ran into

* When deploy project with Vercel, the database will be read-only. Thus if we want to write into database, we will encounter problems. We avoid this problem by going databaseless.
* Limitation of ChatGPT’s token size. We developed an algorithm to solve the problem involving feeding different pieces of passage into ChatGPT and let it do a several-layer summarization.

## Accomplishments that we're proud of

* Pause the video, ask a question, a “peopleless” intelligent will answer your question! Wouldn’t that be exciting enough?

## What we learned

* How Vercel interact with database
* How to regulate ChatGPT’s API

## What's next for Xplain

### More features

* Search in the video. Now video platforms you cannot really search to certain concise frame. With our platform, users will be able to search in a video.
* Personalization in further steps: We will build databases for each users upon which we could generate more personalized report for users.
* More than words: Now our platform is answering the question totally based on words. Yet next step we can make it generate a series of animation to better explain the question

### Wider usage

* Other than YouTube videos, we will extend our platform to other video platforms. With our translation feature, we can extend it different languages. Also, we can also extend our project, with whisper, to any other video platform without subtitle, like Edx, Udemy, etc.
* Furthermore, other than only using the transcript as input, we will also try to implement our own “Large Video Model”, making our model understand from more abundant and deeper dimension.
* Even better, we could extend our project to non-video form of information. For example, we can do a PDF helper for those who read a lot. We can also extend, with image recognition, our platform to live videos, people’s lives, even directly interact with people’s brain instead of through words.
## Inspiration

it's really fucking cool that big LLMs (ChatGPT) are able to figure out on their own how to use various tools to accomplish tasks.

for example, see Toolformer: Language Models Can Teach Themselves to Use Tools (<https://arxiv.org/abs/2302.04761>)

this enables a new paradigm self-assembling software: machines controlling machines.

what if we could harness this to make our own lives better -- a lil LLM that works for you?

## What it does

i made an AI assistant (SMS) using GPT-3 that's able to access various online services (calendar, email, google maps) to do things on your behalf.

it's just like talking to your friend and asking them to help you out.

## How we built it

a lot of prompt engineering + few shot prompting.

## What's next for jarbls

shopping, logistics, research, etc -- possibilities are endless

* more integrations !!!
the capabilities explode exponentially with the number of integrations added
* long term memory

come by and i can give you a demo