## Inspiration

According to the World Health Organization (WHO), 1 in every 5 college students suffer from mental health disorders including depression and anxiety. This epidemic has far-reaching personal and societal consequences – increasing disease risk, fracturing relationships, and reducing workforce productivity.

Current methods of depression diagnosis are extremely time-consuming, relying heavily on one-on-one clinical interviews conducted manually by psychiatrists. Moreover, the subjective nature of evaluations often lead to inconsistent medical advice and recommended treatment paths across different specialists. As a result, there is a clear need for a solution for efficient and standardized depression diagnosis.

## What it does

SmartPsych is a streamlined web platform for automated depression diagnosis from videos of natural patient conversation. Leveraging a unique mix of audio, computer vision, and natural language processing deep learning algorithms, SmartPsych accurately pinpoints and quantifies the complex network of symptoms underlying depression for further psychiatric evaluation. Furthermore, SmartPsych has a special sentence-by-sentence playback feature that enables psychiatrists to hone in on specific sentences (color-coded for severity from green to red being the most severe). This enables psychiatrists to identify the patient-specific struggles and affected topics – paving the way for personalized treatment.

SmartPsych is currently able to detect depression sentiment from patient words, negative emotions (e.g. anger, contempt, fear, sadness) from facial images, and valence and arousal (i.e. positive energy levels) from spectral qualities of a user's voice.

Overall, SmartPsych represents an entirely new way of depression diagnosis – instead of wasting the valuable time of psychiatrists, we delegate the tedious task of identifying depression symptoms to machines and bring in psychiatrists at the end for the final diagnosis and treatment when their expertise is most crucial. This hybrid man and machine model increases efficiency and retains accuracy compared to relying on machine learning predictions alone.

## How I built it

The web application was built entirely in Flask; to facilitate user interaction with the application, I used the pre-designed UIKit components to create a minimalistic yet user-friendly front-end.

Since current sentiment analysis methods are unable to directly infer depression, I trained my own CNN-LSTM neural network on the 189 interview transcripts of both depressed and non-depressed patients from the USC Distress Analysis Wizard-of-Oz database. After undergoing supervised learning, my neural network had a validation accuracy of 69%, which is highly comparable to current human inter-rater variabilities between 70-75%.

Emotion classification and facial landmark detection was performed through the Microsoft Cognitive Face API, valence and arousal prediction was performed through multi-variate regression on audio Mel Frequency Cepstral Coefficients (MFCC), and speech-to-text transcription was performed through Google Cloud.

## Accomplishments that I'm proud of

Integrating custom-trained Keras/Tensorflow models, Google and Microsoft cloud APIs, and regression trees into a single unified and streamlined application.

## What I learned

The sheer amount of factors behind depression, and calling cloud APIs!

## What's next for SmartPsych

While SmartPsych can now be easily used on the web for effective depression diagnosis, there is still much room left for improvement. Development of a mobile form of SmartPsych to increase usability, integration of other biometrics (i.e. sleep quality, heart rate, exercise) into depression diagnosis, and increasing understanding of depression sentiment prediction with entity analysis.
## ✨ Inspiration

Driven by the goal of more accessible and transformative education, our group set out to find a viable solution. Stocks are very rarely taught in school and in 3rd world countries even less, though if used right, it can help many people go above the poverty line. We seek to help students and adults learn more about stocks and what drives companies to gain or lower their stock value and use that information to make more informed decisions.

## 🚀 What it does

Users are guided to a search bar where they can search a company stock for example "APPL" and almost instantly they can see the stock price over the last two years as a graph, with green and red dots spread out on the line graph. When they hover over the dots, the green dots explain why there is a general increasing trend in the stock and a news article to back it up, along with the price change from the previous day and what it is predicted to be from. An image shows up on the side of the graph showing the company image as well.

## 🔧 How we built it

When a user writes a stock name, it accesses yahooFinance API and gets the data on stock price from the last 3 years. It takes the data and converts to a JSON File on local host 5000. Then using flask it is converted to our own API that populates the ChartsJS API with the data on the stock. Using a Matlab Server, we then take that data to find areas of most significance (the absolute value of slope is over a certain threshold). Those data points are set as green if it is positive or red if it is negative. Those specific dates in our data points are fed back to Gemini and asks it why it is thinking the stock shifted as it did and the price changed on the day as well. The Gemini at the same time also takes another request for a phrase that easy for the json search API to find a photo for about that company and then shows it on the screen.

## 🤯 Challenges we ran into

Using the amount of API's we used and using them properly was VERY Hard especially making our own API and incorporating Flask. As well, getting Stock data to a MATLAB Server took a lot of time as it was all of our first time using it. Using POST and Fetch commands were new for us and took us a lot of time for us to get used too.

## 🏆 Accomplishments that we're proud of

Connecting a prompt to a well-crafted stocks portfolio.
learning MATLAB in a time crunch.
connecting all of our API's successfully
making a website that we believe has serious positive implications for this world

## 🧠 What we learned

MATLAB integration
Flask Integration
Gemini API

## 🚀What's next for StockSee

* Incorporating it on different mediums such as VR so users can see in real-time how stocks shift in from on them in an interactive way.
* Making a small questionnaire on different parts of the stocks to ask whether if it good to buy it at the time
* Use (MPT) and other common stock buying algorthimns to see how much money you would have made using it.
## Inspiration

According to APA, in 2018, 39% of researched Americans reported that they don't go to therapists because they could not afford the therapy cost, time, and daycare. Our team believes that after the pandemic and also the recent high rate of price inflation, such concern is more alert. "Follow Up" is an attempt to lower the therapy cost. We aimed at reducing expensive in-person therapy sessions with accurate remote AI-powered diagnoses, requiring less therapist input and more flexibility for the patients. For now, we are focusing on determining *depression*.

## What it does

"Follow Up" creates a platform for therapists and patients to communicate after a traditional in-person session:

### - From the patient:

They will be asked to answer a questionnaire provided by the therapists, and also upload the video URLs where they answer the questions.

### - From the therapist:

After a traditional therapy session, the therapist will jot down some relevant questions. Those questions can be some self-report questions regarding how the patient feels.
The therapist will also require their patients to upload a video answering some critical questions, which they think will best reflect the patient's mental state. We introduce *Hume* to help us detect the emotion based on their video answer.
The 5 emotions we used to suggest that a person has depression were: *Anxiety, Guilt, Tiredness, Sadness, and Distress*. We planned to create a model to train how much each emotion weighs in terms of reflecting depression.

## How we built it

We used *Flask, HTML, and CSS* mainly to create the full-stack development. We set up the database using *CockroachDB* serverless, and utilize *Hume* to help us analyze the emotion from the uploaded video.

## Challenges we ran into

Honestly, just try to make everything run. The first problem we had was connecting the local machine to the *CockroachDB*, as we did not have *PostgresDB* ready. Secondly, we could not just feed any URL to the *Hume* model, so we tried to incorporate using Google Cloud API for that. We could not find any data regarding how much each emotion relates to depression, so the emotions' weights are untrained. Lastly, we also have some problems with the *Flask* backend.

## Accomplishments that we're proud of

This is our first hackathon so we were quite lost in brainstorming the idea at the beginning, not to mention that one teammate disappeared unannouncedly. We are proud that we stayed until the end and did not give up.

## What we learned

I personally never searched "What is" or "How to" in 24 hours this much in my life. So, in other words, a lot.

## What's next for Follow Up

We believe this has a lot of potential. This current version only focuses on *depression*, but there are a lot more symptoms we can analyze. Also, because of the lack of training data for the weight of each emotion, we believe that more research on how much of each dominant emotion is reflected in mentally ill patients will put our idea into more multi-dimensional use.

Some research links:
[link](https://www.verywellmind.com/depression-statistics-everyone-should-know-4159056)
[link](https://www.apa.org/monitor/2020/07/datapoint-care)

Pictures are license-free from Pinterest\*