resonate / docs /OLD_README.md
sartajbhuvaji's picture
Added files from branch huggingface
5f773d1 verified

A newer version of the Streamlit SDK is available: 1.43.2

Upgrade

Resonate

Current Phase: Sprint 1

Project Overview

Resonate is a Retrieval-augmented generation (RAG) powered Large Language Model application that helps you chat with your meetings to answer questions and generate insights.

Objectives

  • User should be able to upload an audio/video meeting file along with a meeting Topic
  • There can be multiple meeting topics. With each topic having a series of meetings.
  • Use would then be able to choose a topic and chat with the meeting just and ask any question

Initial Sketches

RAG Inference

  • The user would select the meeting Topic and ask a question.
  • Pinecone would retrieve relevant information and would feed the LLm with custom prompt, context, and the user query.
  • We also plan to add a Semantic Router to route queries according to the user input.
  • The LLm would then generate the result and answer the question.

image

Data Store

  • The below diagram shows how we plan to store data using Pinecone which is a popular Vector DB.
  • User would upload meetings in audio/video format.
  • We would use AWS Transcribe to diarize and transcribe the audio file into timestamp, speaker, text (this is simplified)
  • We would embed the text data into vectors that would be uploaded to Pinecone serverless.

image

Research

  • We would try multiple Vector embeddings and also fine-tune LLM Models using Microsoft DeepSpeed on the custom dataset and compare the performance of these models. image

Proposed UI

  • Below is the sketch of proposed UI. image