Spaces:
Running
Running
| # Resonate | |
| #### Current Phase: Sprint 1 | |
| ## Project Overview | |
| Resonate is a Retrieval-augmented generation (RAG) powered Large Language Model application that helps you chat with your meetings to answer questions and generate insights. | |
| ## Objectives | |
| - User should be able to upload an audio/video meeting file along with a meeting `Topic` | |
| - There can be multiple meeting topics. With each topic having a series of meetings. | |
| - Use would then be able to choose a `topic` and chat with the meeting just and ask any question | |
| ## Initial Sketches | |
| RAG Inference | |
| - The user would select the meeting `Topic` and ask a question. | |
| - Pinecone would retrieve relevant information and would feed the LLm with custom prompt, context, and the user query. | |
| - We also plan to add a `Semantic Router` to route queries according to the user input. | |
| - The LLm would then generate the result and answer the question. | |
|  | |
| Data Store | |
| - The below diagram shows how we plan to store data using `Pinecone` which is a popular Vector DB. | |
| - User would upload meetings in audio/video format. | |
| - We would use `AWS Transcribe` to diarize and transcribe the audio file into `timestamp, speaker, text` (this is simplified) | |
| - We would embed the text data into vectors that would be uploaded to Pinecone serverless. | |
|  | |
| Research | |
| - We would try multiple `Vector embeddings` and also fine-tune `LLM Models` using `Microsoft DeepSpeed` on the custom dataset and compare the performance of these models. | |
|  | |
| Proposed UI | |
| - Below is the sketch of proposed UI. | |
|  | |