## Inspiration

Memory athletes retain large amounts of information using mnemonics, story-telling, and visualizations. Picture Pathway aims to emulate this studying methodology and bring it into the classroom!

## What it does

Picture Pathway is a student-teacher platform. Teachers submit the problem they would like their class to visualize and/or convert into a story. From there, the student describes a scene to DALL-E and then receives a generated image to add to their story on solving their assigned problems

In our example, a teacher is looking to solidify the process of Integration for her students; thus, they have assigned a series of steps to 'storify'. The text contained in yellow represents what a student user's responses might look like (and our last slide demonstrates what the corresponding image output may be).

## How we built it

-Front-End: Repl.it - HTML, Javascript
-Back-End: Python (Jango), SQLite

## Challenges we ran into

-Most of our members are just beginning their coding journey so there was certainly a learning curve!
-The integration of Dall-E API was especially uncharted territory for our team and required much research to implement
-Debugging（πーπ）

## Accomplishments that we're proud of

Our team is most proud of our ability to riff off each other--- most of us met for the first time just Friday, yet we trusted one another to perform our assigned roles and successfully worked our way from 0 to a working prototype

## What we learned

-3/4 members learned Django + SQL for the first time!
-APIs can interact on the backend (what enabled us to pull images from Dall-E to embed into our project!)

## What's next for Picture Pathway

-All our members are passionate about accessibility in STEM education
## Inspiration

There are millions of people around the world who have a physical or learning disability which makes creating visual presentations extremely difficult. They may be visually impaired, suffer from ADHD or have disabilities like Parkinsons. For these people, being unable to create presentations isn’t just a hassle. It’s a barrier to learning, a reason for feeling left out, or a career disadvantage in the workplace. That’s why we created **Pitch.ai.**

## What it does

Pitch.ai is a web app which creates visual presentations for you as you present. Once you open the web app, just start talking! Pitch.ai will listen to what you say and in real-time and generate a slide deck based on the content of your speech, just as if you had a slideshow prepared in advance.

## How we built it

We used a **React** client combined with a **Flask** server to make our API calls. To continuously listen for audio to convert to text, we used a react library called “react-speech-recognition”. Then, we designed an algorithm to detect pauses in the speech in order to separate sentences, which would be sent to the Flask server.

The Flask server would then use multithreading in order to make several API calls simultaneously. Firstly, the **Monkeylearn** API is used to find the most relevant keyword in the sentence. Then, the keyword is sent to **SerpAPI** in order to find an image to add to the presentation. At the same time, an API call is sent to OpenAPI’s GPT-3 in order to generate a caption to put on the slide. The caption, keyword and image of a single slide deck are all combined into an object to be sent back to the client.

## Challenges we ran into

* Learning how to make dynamic websites
* Optimizing audio processing time
* Increasing efficiency of server

## Accomplishments that we're proud of

* Made an aesthetic user interface
* Distributing work efficiently
* Good organization and integration of many APIs

## What we learned

* Multithreading
* How to use continuous audio input
* How to use React hooks, Animations, Figma

## What's next for Pitch.ai

* Faster and more accurate picture, keyword and caption generation
* "Presentation mode”
* Integrate a database to save your generated presentation
* Customizable templates for slide structure, color, etc.
* Build our own web scraping API to find images
**What inspired us**
Despite the prevalence of LLMs increasing, their power still hasn't been leveraged to improve the experience of students during class. In particular, LLMs are often discouraged by professors, often because they often give inaccurate or too much information. To remedy this issue, we created an LLM that has access to all the information for a course, including the course information, lecture notes, and problem sets. Furthermore, in order for this to be useful for actual courses, we made sure for the LLM to not answer specific questions about the problem set. Instead, the LLM guides the student and provides relevant information for the student to complete the coursework without providing students with the direct answer. This essentially serves as a TA for students to help them navigate their problem sets.

**What we learned**
Through this project, we delved into the complexities of integrating AI with software solutions, uncovering the essential role of user interface design and the nuanced craft of prompt engineering. We learned that crafting effective prompts is crucial, requiring a deep understanding of the AI’s capabilities and the project's specific needs. This process taught us the importance of precision and creativity in prompt engineering, where success depends on translating educational objectives into prompts that generate meaningful AI responses.

Our exploration also introduced us to the concept of retrieval-augmented generation (RAG), which combines the power of information retrieval with generative models to enhance the AI's ability to produce relevant and contextually accurate outputs. While we explored the potentials of using the OpenAI and Together APIs to enrich our project, we ultimately did not incorporate them into our final implementation. This exploration, however, broadened our understanding of the diverse AI tools available and their potential applications. It underscored the importance of selecting the right tools for specific project needs, balancing between the cutting-edge capabilities of such APIs and the project's goals. This experience highlighted the dynamic nature of AI project development, where learning about and testing various tools forms a foundational part of the journey, even if some are not used in the end.

**How we built our project**
Building our project required a strategic approach to assembling a comprehensive dataset from the Stanford CS 106B course, which included the syllabus, problem sets, and lectures. This effort ensured our AI chatbot was equipped with a detailed understanding of the course's structure and content, setting the stage for it to function as an advanced educational assistant. Beyond the compilation of course materials, a significant portion of our work focused on refining an existing chatbot user interface (UI) to better serve the specific needs of students engaging with the course. This task was far from straightforward; it demanded not only a deep dive into the chatbot's underlying logic but also innovative thinking to reimagine how it interacts with users. The modifications we made to the chatbot were extensive and targeted at enhancing the user experience by adjusting the output behavior of the language learning model (LLM).
A pivotal change involved programming the chatbot to moderate the explicitness of its hints in response to queries about problem sets. This adjustment required intricate tuning of the LLM's output to strike a balance between guiding students and stimulating independent problem-solving skills. Furthermore, integrating direct course content into the chatbot’s responses necessitated a thorough understanding of the LLM's mechanisms to ensure that the chatbot could accurately reference and utilize the course materials in its interactions. This aspect of the project was particularly challenging, as it involved manipulating the chatbot to filter and prioritize information from the course data effectively. Overall, the effort to modify the chatbot's output capabilities underscored the complexity of working with advanced AI tools, highlighting the technical skill and creativity required to adapt these systems to meet specific educational objectives.

**Challenges we faced**
Some challenges we faced included scoping our project to ensure that it is feasible given the constraints we had for this hackathon including time. We learned React.js and PLpgSQL for our project since we had only used JavaScript previously. Other challenges we faced were installing Docker, Supabase CLI, and ensuring all dependencies are properly managed. Moreover, we also had to configure Supabase and create the database schema. There were also deployment configuration issues as we had to integrate our front-end application with our back-end to ensure that they are communicating properly.