# better.me, AI Journaling

## Project Description

better.me is an AI journaling tool that helps you analyze your emotions and provide you with smart recommendations for your well being. We used NLP emotion analytics to process text data and incorporated a suicidal prevention algorithm that will help you make better informed decisions about your mental health.

## Motivation

Poor mental health is a growing pandemic that is still being stigmatized. Even after spending $5 Billion in federal investments for mental health, 1.3 million adults attempted suicide and 1.1 million plans to commit suicide.

>
> Our mission is to provide a private environment to help people analyze their emotions and receive mental health support.
>
>
>

## MVP Product Features Overview

| Features | Description |
| --- | --- |
| Personal Journal | Better Me is a personal AI-powered journal where users can write daily notes reflecting on their life's progress. |
| NLP Emotion Analytics | With the help of natural language process, Better Me will classify the user's emotional situation and keep a track of the data. |
| Smart Recommendations | It uses this monitored data to suggest appropriate mental health resources to the users and also provides them with suitable data analytics. |
| Suicide Prevention | In order to take a step forward towards suicide prevention, it also incorporates a suicidal text detection algorithm that triggers a preventive measure . |

## How we built it

We used Google T5 NLP model for emotional recognition and categorizing emotions. We trained data set with deep learning to develop a fine tuned BERT model to prevent suicide. We also implemented our own algorithm to make resource recommendations to users based on their emotional changes, and also did some data analytics. Due to time restraints and a member's absence, we had to change from React.js plus Firebase stack to Streamlit, a python library design framework.

## Challenges

Initially, we tried creating a dashboard using full-stack web development, however, it proved to be quite a challenging task with the little amount of time we had. We decided to shift our focus to quickly prototyping using a lightweight tool, and streamlit was the ideal choice for our needs. While deploying our suicide prevention algorithm on Google Cloud Function, we had trouble deploying due to memory availability constraints.

## Accomplishments

We are proud that we came up with such a novel idea that could be useful to innumerable people suffering from mental health issues, or those who are like to stay reserved with themselves or in a confused state about their mental well-being, just by writing about their daily lives. We are also proud of incorporating a suicide prevention algorithm, which could be life-saving for many.

## Roadmap

| Future Implementations | Description |
| --- | --- |
| Firebase Back End Architecture | We hope to design a scalable backend which accommodates for the users needs. |
| AI Mental Health Chat bot | Provide on the spot, mental health support using Dialogflow AI chat bot. |
| Connect with Therapists | Elevate data analytical features to connect and report to personal therapists. |
| Scaling Up | Fund our project and develop this project with scalable front and back end. |
| Languages Support | Support multiple languages, including French, Mandarin, and Spanish. |
## Inspiration

COVID revolutionized the way we communicate and work by normalizing remoteness. **It also created a massive emotional drought -** we weren't made to empathize with each other through screens. As video conferencing turns into the new normal, marketers and managers continue to struggle with remote communication's lack of nuance and engagement, and those with sensory impairments likely have it even worse.

Given our team's experience with AI/ML, we wanted to leverage data to bridge this gap. Beginning with the idea to use computer vision to help sensory impaired users detect emotion, we generalized our use-case to emotional analytics and real-time emotion identification for video conferencing in general.

## What it does

In MVP form, empath.ly is a video conferencing web application with integrated emotional analytics and real-time emotion identification. During a video call, we analyze the emotions of each user sixty times a second through their camera feed.

After the call, a dashboard is available which displays emotional data and data-derived insights such as the most common emotions throughout the call, changes in emotions over time, and notable contrasts or sudden changes in emotion

**Update as of 7.15am: We've also implemented an accessibility feature which colors the screen based on the emotions detected, to aid people with learning disabilities/emotion recognition difficulties.**

**Update as of 7.36am: We've implemented another accessibility feature which uses text-to-speech to report emotions to users with visual impairments.**

## How we built it

Our backend is powered by Tensorflow, Keras and OpenCV. We use an ML model to detect the emotions of each user, sixty times a second. Each detection is stored in an array, and these are collectively stored in an array of arrays to be displayed later on the analytics dashboard. On the frontend, we built the video conferencing app using React and Agora SDK, and imported the ML model using Tensorflow.JS.

## Challenges we ran into

Initially, we attempted to train our own facial recognition model on 10-13k datapoints from Kaggle - the maximum load our laptops could handle. However, the results weren't too accurate, and we ran into issues integrating this with the frontend later on. The model's still available on our repository, and with access to a PC, we're confident we would have been able to use it.

## Accomplishments that we're proud of

We overcame our ML roadblock and managed to produce a fully-functioning, well-designed web app with a solid business case for B2B data analytics and B2C accessibility.

And our two last minute accessibility add-ons!

## What we learned

It's okay - and, in fact, in the spirit of hacking - to innovate and leverage on pre-existing builds. After our initial ML model failed, we found and utilized a pretrained model which proved to be more accurate and effective.

Inspiring users' trust and choosing a target market receptive to AI-based analytics is also important - that's why our go-to-market will focus on tech companies that rely on remote work and are staffed by younger employees.

## What's next for empath.ly

From short-term to long-term stretch goals:

* We want to add on AssemblyAI's NLP model to deliver better insights. For example, the dashboard could highlight times when a certain sentiment in speech triggered a visual emotional reaction from the audience.
* We want to port this to mobile and enable camera feed functionality, so those with visual impairments or other difficulties recognizing emotions can rely on our live detection for in-person interactions.
* We want to apply our project to the metaverse by allowing users' avatars to emote based on emotions detected from the user.
## Inspiration

Mental health is a serious issue but also one hard to diagnose and address. In these times of our virtual environments and uncertainties, people tend to face more difficulty and strain on their mental health. We aim to help individuals become more aware and hopefully, identify signs of mental health issues.

We realized that not too many people look after their mental health until it's far too late. As such, we decided to implement a solution that motivated people to write down their thoughts so that they could track their mood over time. Other mental health tracking apps exist on the market right now, but none utilize NLP to take advantage of the actual writing part of journalling. That is how our idea of Mood was born!

## What it does

Mood is a journaling app. It allows the user to make quick daily entries about how they feel and what significant events occurred. Using this text entry, the app performs sentiment analysis to rate the current mood of the user and tracks it over time. This enables the app to identify trends in the user's mental health daily, weekly, monthly, and overall. It also calculates the most frequently occurring emotional keywords and general keywords throughout a period, on top of the top positive and negative keywords that make the user smile and cry the most.

## How we built it

We built the interface of Mood using Flutter. Our team decided that for a journaling application, it would be more accessible if it was available on mobile. The sentiment analysis is done using Python and its NLP libraries like textblob, spacy, nltk, and transformers. Since the backend is written in Python, we decided to use Flask to manage our back-end server. These are connected with a REST API.

## Challenges we ran into

We all had to learn a new technology for this project and was definitely one of the biggest challenges that we faced. Another big challenge that we ran into is creating APIs to connect our front-end to the back-end.

## Accomplishments that we're proud of

Our team has a lot to be proud of. We all learned something new and got something to take away from this Hackathon. Even though we stayed up really late, we are happy that we have something to show to the judges.

## What we learned

One of the biggest lessons that we learned from this experience is how important the initial development plan is. Running into problems later on caused our team to spend more time to find a workaround to problems that in hindsight could have been better planned for.

## What's next for Mood

Mood has lots of room for future improvement. Features such as user authentication and cloud storage would be great additions to improve the security and protect the privacy of our users. Also, notification systems on long-term and short-term abnormal trends (bipolar, depression,...) of mood would be meaningful for users to understand their emotional behaviour. Other improvements would include a more accurate and advanced model of sentimental analysis and improving accessibility of the interface (i.e. voice or image input).
An online system that learns the users’ mood patterns and responds better to changes.