![](https://res.cloudinary.com/devpost/image/fetch/s--SLH6Xr3g--/c_limit,f_auto,fl_lossy,q_auto:eco,w_900/v1/logistics/demo.gif)

## Inspiration

📚In our current knowledge economy, our information is also our most important valuable commodity.

💡 Knowledge is available in almost infinite abundance 📈, delivered directly through our digital devices 📱 💻 , the world is more connected and educated than ever across the globe 🌎 🌍 🌏. However, the surge of information draws adverse effects💥 🔥 🌈! With information circulating as rapid as ever, information and cognitive overload 🧠👎🏼 is a present symptom amongst our lives.

✨💡✨Mr. Goose 🦢 is here to help by aggregating millions of sources to simplify complex concepts into comprehensible language for even a five-year-old. ✨💡✨

## What it does

It is a chrome extension for users to conveniently type in questions, 💡 highlight 💡 paragraphs, sentences, or words on their browser, and receive a ⭐️simple to understand answer or explanation. 🎇 🎆

## How we built it

✨🔨Our chrome extension was built using JavaScript, HTML, and CSS, using Rest API. ✨As for the backend, functions are deployed on Google Cloud Functions ☁️☁️☁️and calls the Google Cloud Language API☁️☁️☁️, which uses Natural Language Processing 💬 💡 to figure out what entities are in the highlighted text. Once we’ve figured out what the text is about, we use it to parse the web using APIs such as the Reddit API, the StackOverflow/Stack Exchange API, and the Wikipedia API. ⭐️⭐️⭐️

## Challenges we ran into

One of the 💪 main challenges 💪 we ran into was while building 🌼👩🏼‍💻 🌻 the wireframes of the extension, discussing 💭💭 and re-evaluating the logic of the app’s uses. ✨ As we were 🔨 designing 🧩 several features, we tried to discuss what features would be the most user-friendly and important while also maintaining the efficiency 📈 📈 📈 and importance of learning/knowledge 📚of our Chrome extension. ✨

## Accomplishments that we're proud of

✨✨✨We were extremely proud ⭐️ of the overall layout and elements 🧩🧩🧩 we implemented into our app design, such as the animated goose 🦢 that one of our team members drew and animated from scratch. From the color 🔴 🟠 🟡 choices to the attention to details like which words 💬 📃 📄 should be important in the NLP API to the resulted information 📊, we had to take a lot into consideration for our project, and it truly was a fun learning experience. 👍👍👍

## What we learned

🌟 How to create a Chrome Extension
🌟 How to use Google Firebase
🌟 How to use Google Cloud's NLP API, Stack Exchange API, Reddit API, Wikipedia API
🌟 How to integrate all of these together
🌟 How to create animated images for implementation on the extensions

## What's next for Mr. Goose

✨Adapting our extensions compatibility with other browsers.
✨Adding a voice recognition feature to allow users to ask questions and receive simplified answers in return
✨Adding ability to access images while on the extension
## Inspiration

The health industry challenge got us thinking about prescriptions, how unreliable our current system is and how we lack the ability to track a patient's history. Tracking medical history is often crucial in recognizing the signs and symptoms of a medical condition.

## What it does

Provides a user access to their current prescriptions and prescription history.
Allows user to authenticate their doctors, nurses and pharmacists using a QR code. This gives the patient autonomy over their own data.

## How we built it

Mobile application (Flutter) : Allows patient to view own history, print out prescriptions, etc.
Web application (HTML/CSS/JS) : Doctor or pharmacist will generate a QR code. Patient scans the QR code, authenticating the professional, and the professional will be automatically redirected to user's profile (after first use, the professional will remain authenticated until user removes authorization). Doctor can then fill out prescription, which can be accessed by patient and pharmacist.
Database (Firebase): Database to contain user accounts - three account types: Medical professional (can write prescriptions), Pharmaceutical professional (can fill prescriptions) and Patient (administrator, accesses their own history and who has access to it)

## Challenges we ran into

Choosing the right database structure.
Communicating virtually is also a barrier.

## Accomplishments that we're proud of

Potentially an integrated system that would change how medical data is approached.

## What we learned

It is worth taking the time to think of an idea worth doing. This was not the idea we originally were going to go with.

## What's next for Pocket Prescription

We implemented basic requirements. We plan to continue the project, the next step being researching security methods.
## Inspiration

Has your browser ever looked like this?

![](https://i.imgur.com/waCM1K0.png)
... or this?
![](https://i.imgur.com/WX2dTfz.png)

Ours have, *all* the time.

Regardless of who you are, you'll often find yourself working in a browser on not just one task but a variety of tasks. Whether its classes, projects, financials, research, personal hobbies -- there are many different, yet predictable, ways in which we open an endless amount of tabs for fear of forgetting a chunk of information that may someday be relevant.

Origin aims to revolutionize your personal browsing experience -- one workspace at a time.

## What it does

In a nutshell, Origin uses state-of-the-art **natural language processing** to identify personalized, smart **workspaces**. Each workspace is centered around a topic comprised of related tabs from your browsing history, and Origin provides your most recently visited tabs pertaining to that workspace and related future ones, a generated **textual summary** of those websites from all their text, and a **fine-tuned ChatBot** trained on data about that topic and ready to answer specific user questions with citations and maintaining history of a conversation. The ChatBot not only answers general factual questions (given its a foundation model), but also answers/recalls specific facts found in the URLs/files that the user visits (e.g. linking to a course syllabus).

Origin also provides a **semantic search** on resources, as well as monitors what URLs other people in an organization visit and recommend pertinent ones to the user via a **recommendation system**.

For example, a college student taking a History class and performing ML research on the side would have sets of tabs that would be related to both topics individually. Through its clustering algorithms, Origin would identify the workspaces of "European History" and "Computer Vision", with a dynamic view of pertinent URLs and widgets like semantic search and a chatbot. Upon continuing to browse in either workspace, the workspace itself is dynamically updated to reflect the most recently visited sites and data.

**Target Audience**: Students to significantly improve the education experience and industry workers to improve productivity.

## How we built it

![](https://i.imgur.com/HYsZ3un.jpg)

**Languages**: Python ∙ JavaScript ∙ HTML ∙ CSS

**Frameworks and Tools**: Firebase ∙ React.js ∙ Flask ∙ LangChain ∙ OpenAI ∙ HuggingFace

There are a couple of different key engineering modules that this project can be broken down into.

### 1(a). Ingesting Browser Information and Computing Embeddings

We begin by developing a Chrome Extension that automatically scrapes browsing data in a periodic manner (every 3 days) using the Chrome Developer API. From the information we glean, we extract titles of webpages. Then, the webpage titles are passed into a pre-trained Large Language Model (LLM) from Huggingface, from which latent embeddings are generated and persisted through a Firebase database.

### 1(b). Topical Clustering Algorithms and Automatic Cluster Name Inference

Given the URL embeddings, we run K-Means Clustering to identify key topical/activity-related clusters in browsing data and the associated URLs.

We automatically find a description for each cluster by prompt engineering an OpenAI LLM, specifically by providing it the titles of all webpages in the cluster and requesting it to output a simple title describing that cluster (e.g. "Algorithms Course" or "Machine Learning Research").

### 2. Web/Knowledge Scraping

After pulling the user's URLs from the database, we asynchronously scrape through the text on each webpage via Beautiful Soup. This text provides richer context for each page beyond the title and is temporarily cached for use in later algorithms.

### 3. Text Summarization

We split the incoming text of all the web pages using a CharacterTextSplitter to create smaller documents, and then attempt a summarization in a map reduce fashion over these smaller documents using a LangChain summarization chain that increases the ability to maintain broader context while parallelizing workload.

### 4. Fine Tuning a GPT-3 Based ChatBot

The infrastructure for this was built on a recently-made popular open-source Python package called **LangChain** (see <https://github.com/hwchase17/langchain>), a package with the intention of making it easier to build more powerful Language Models by connecting them to external knowledge sources.

We first deal with data ingestion and chunking, before embedding the vectors using OpenAI Embeddings and storing them in a vector store.

To provide the best chat bot possible, we keep track of a history of a user's conversation and inject it into the chatbot during each user interaction while simultaneously looking up relevant information that can be quickly queries from the vector store. The generated prompt is then put into an OpenAI LLM to interact with the user in a knowledge-aware context.

### 5. Collaborative Filtering-Based Recommendation

Provided that a user does not turn privacy settings on, our collaborative filtering-based recommendation system recommends URLs that other users in the organization have seen that are related to the user's current workspace.

### 6. Flask REST API

We expose all of our LLM capabilities, recommendation system, and other data queries for the frontend through a REST API served by Flask. This provides an easy interface between the external vendors (like LangChain, OpenAI, and HuggingFace), our Firebase database, the browser extension, and our React web app.

### 7. A Fantastic Frontend

Our frontend is built using the React.js framework. We use axios to interact with our backend server and display the relevant information for each workspace.

## Challenges we ran into

1. We had to deal with our K-Means Clustering algorithm outputting changing cluster means over time as new data is ingested, since the URLs that a user visits changes over time. We had to anchor previous data to the new clusters in a smart way and come up with a clever updating algorithm.
2. We had to employ caching of responses from the external LLMs (like OpenAI/LangChain) to operate under the rate limit. This was challenging, as it required revamping our database infrastructure for caching.
3. Enabling the Chrome extension to speak with our backend server was a challenge, as we had to periodically poll the user's browser history and deal with CORS (Cross-Origin Resource Sharing) errors.
4. We worked modularly which was great for parallelization/efficiency, but it slowed us down when integrating things together for e2e testing.

## Accomplishments that we're proud of

The scope of ways in which we were able to utilize Large Language Models to redefine the antiquated browsing experience and provide knowledge centralization.

This idea was a byproduct of our own experiences in college and high school -- we found ourselves spending significant amounts of time attempting to organize tab clutter systematically.

## What we learned

This project was an incredible learning experience for our team as we took on multiple technically complex challenges to reach our ending solution -- something we all thought that we had a potential to use ourselves.

## What's next for Origin

We believe Origin will become even more powerful at scale, since many users/organizations using the product would improve the ChatBot's ability to answer commonly asked questions, and the recommender system would perform better in aiding user's education or productivity experiences.