Spaces:
Sleeping
Sleeping
File size: 1,931 Bytes
5f773d1 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 |
# Resonate
#### Current Phase: Sprint 1
## Project Overview
Resonate is a Retrieval-augmented generation (RAG) powered Large Language Model application that helps you chat with your meetings to answer questions and generate insights.
## Objectives
- User should be able to upload an audio/video meeting file along with a meeting `Topic`
- There can be multiple meeting topics. With each topic having a series of meetings.
- Use would then be able to choose a `topic` and chat with the meeting just and ask any question
## Initial Sketches
RAG Inference
- The user would select the meeting `Topic` and ask a question.
- Pinecone would retrieve relevant information and would feed the LLm with custom prompt, context, and the user query.
- We also plan to add a `Semantic Router` to route queries according to the user input.
- The LLm would then generate the result and answer the question.
Data Store
- The below diagram shows how we plan to store data using `Pinecone` which is a popular Vector DB.
- User would upload meetings in audio/video format.
- We would use `AWS Transcribe` to diarize and transcribe the audio file into `timestamp, speaker, text` (this is simplified)
- We would embed the text data into vectors that would be uploaded to Pinecone serverless.
Research
- We would try multiple `Vector embeddings` and also fine-tune `LLM Models` using `Microsoft DeepSpeed` on the custom dataset and compare the performance of these models.
Proposed UI
- Below is the sketch of proposed UI.
|