Spaces:
Sleeping
Sleeping
A newer version of the Streamlit SDK is available:
1.43.2
Resonate
Current Phase: Sprint 1
Project Overview
Resonate is a Retrieval-augmented generation (RAG) powered Large Language Model application that helps you chat with your meetings to answer questions and generate insights.
Objectives
- User should be able to upload an audio/video meeting file along with a meeting
Topic
- There can be multiple meeting topics. With each topic having a series of meetings.
- Use would then be able to choose a
topic
and chat with the meeting just and ask any question
Initial Sketches
RAG Inference
- The user would select the meeting
Topic
and ask a question. - Pinecone would retrieve relevant information and would feed the LLm with custom prompt, context, and the user query.
- We also plan to add a
Semantic Router
to route queries according to the user input. - The LLm would then generate the result and answer the question.
Data Store
- The below diagram shows how we plan to store data using
Pinecone
which is a popular Vector DB. - User would upload meetings in audio/video format.
- We would use
AWS Transcribe
to diarize and transcribe the audio file intotimestamp, speaker, text
(this is simplified) - We would embed the text data into vectors that would be uploaded to Pinecone serverless.
Research
- We would try multiple
Vector embeddings
and also fine-tuneLLM Models
usingMicrosoft DeepSpeed
on the custom dataset and compare the performance of these models.
Proposed UI