pho·bi·a (/ˈfōbēə/); an extreme or **irrational** fear of or **aversion** to something.

## Inspiration

Starting off with an interest in VR/AR applications, our group pondered how we could solve real-world problems with a virtual solution. As we got to know other, we all connected over our shared fear of spiders. We set out to overcome our arachnophobia, which inspired us to create an exposure therapy project aimed at helping users "face their fears."

## What it does

>
> Exposure therapy is a type of therapy in which you're gradually exposed to the things, situations and activities you fear. It can help treat several conditions, like phobias, post-traumatic stress disorder (PTSD) and panic disorder.
>
>
>

An example of exposure therapy would be baseball players training against a high-speed pitching machine. They crank the speed up to the highest setting, forcing themselves to adjust and become desensitized to the fast-moving ball. After seeing enough pitches, they can track and hit the ball more effectively. The same principle applies to exposure therapy: when users are repeatedly exposed to their phobia, they gradually become desensitized, and over time, their fear lessens until it’s no longer overwhelming.

## How we built it

To build our project, we first leveraged Glitch's simple hosting capabilities and its support for WebXR, which allowed us to seamlessly integrate A-Frame, a web framework for building VR scenes. Using A-Frame, we created immersive environments and dynamic entities for our VR experience.

Next, we integrated Pulsoid, a heartbeat monitor app for Android watches, to gather real-time heartbeat data through API calls, which we displayed in the VR world. We sourced spider assets from a library, animated them ourselves, and programmed them to move randomly around the virtual space, always facing the user.

The speed and size of the spiders are dynamically adjusted based on the user's heartbeat, adding an interactive and personal element to the experience.

## Challenges we ran into

* Tried developing with Unity Engine, but it wasn't optimal for our devices
* Running into issues when learning WebXR and A-Frame
* Glitch rate-limiting us from time to time
* Forcing ourselves to look at spiders

## Accomplishments that we're proud of

* Getting a working VR project when none of us had any 3D development or XR experience
* No longer scared of spiders
* Being able to grasp how to use brand new languages and tools fast enough to build something like this
* How to use and develop onto VR devices
* Integrating heart rate to dynamically alter the simulation experience

## What we learned

* How challenging building and connecting 3Ds environments across devices can be
* Different methods to upload a 3D environment to a VR headset
* Working around CORS issues by creating a proxy server

## What's next for PhobiaPhixer

* Adding different levels of exposure ranging from an easy mode to an intense mode
* Creating different types of phobia treatments (e.g Ophidiophobia: Fear of Snakes, Acrophobia: Fear of Heights)
* Incorporate AI to respond with an encouraging message based on the user's heart beat
## 💡Inspiration

* In Canada, every year there are 5.3 million people who feel they need some form of help for their mental health! But ordinary therapies is unfortunately boring and might be ineffective :( Having to deal with numerous patients every day, it also might be difficult for a mental health professional to build a deeper connection with their patient that allows the patient to heal and improve mentally and in turn physically.
* Therefore, we built TheraVibe.VR! A portable professional that is tailored to understand you! TheraVibe significantly improves patients' mental health by gamifying therapy sessions so that patients can heal wherever and with whomever they can imagine!

## 🤖 What it does

* TheraVibe provides professional psychological advisory powered by Cohere's API with the assistance of a RAG!
* It is powered by Starknet for its private and decentralized approach to store patient information!
* To aid the effort of helping more people, TheraVibe also uses Starknet to reward patients with cryptocurrencies effectively in a decentralized network to incentivize consistency in attending our unique "therapy sessions"!

## 🧠 How we built it

* With the base of C# and Unity Engine, we used blockchain technology via the beautiful Starknet API to create and deploy smart contracts to ensure safe storage of a "doctor's" evaluation of a patient's health condition as well as blockchain transactions made to patient in a gamified manner to incentivize future participation and maximize healing!
* For the memory import NextJS web app, we incorporated Auth0 for the security of our users and hosted it with a GoDaddy domain!
* The verbal interaction of the therapist and the user is powered by ElevenLabs and AssemblyAI! The cognitive process of the therapist is given by Cohere's and a RAG!
* To implement the VR project, we developed animation in Unity with C#, and used the platform to build and run our VR project!

## 🧩 Challenges we ran into

* Auth0 helped reveal a cache problem in our program, and so we had to deal with server-side rendered issues in Nextjs last-minute!
* We managed to deal with a support issue when hosting the domain name on GoDaddy and linking it to our Vercel deployment!
* Deploying the C# Unity APP on Meta Quest 2 took 24 hours of our development!

## 🏆 Accomplishments that we're proud of

* First time deploying on MetaQuest2 and building a project with Unity!
* Integrating multiple API's together like AssemblyAI for speech transcription, ElevenLabs for speech generation, Cohere, and the implementation of a RAG through a complex pipeline with minimal latency!

## 🌐What we learned

* How Auth0 requires HTTPS protocol(always fascinating how we don't have to reinvent the wheel for authenticating users!)
* Our first time hosting on GoDaddy(especially a cool project domain!)
* Building and running production-fast pipelines that have minimal latency to maximize user experience!

## 🚀What's next for TheraVibe.VR

* In the exciting journey ahead, TheraVibe.VR aspires to revolutionize personalized therapy by reducing latency, expanding our immersive scenes, and introducing features like virtual running. Our future goals include crafting an even more seamless and enriching experience and pushing the boundaries of therapeutic possibilities for all our users.
## Inspiration

We all deal with nostalgia. Sometimes we miss our loved ones or places we visited and look back at our pictures. But what if we could revolutionize the way memories are shown? What if we said you can relive your memories and mean it literally?

## What it does

retro.act takes in a user prompt such as "I want uplifting 80s music" and will then use sentiment analysis and Cohere's chat feature to find potential songs out of which the user picks one. Then the user chooses from famous dance videos (such as by Michael Jackson). Finally, we will either let the user choose an image from their past or let our model match images based on the mood of the music and implant the dance moves and music into the image/s.

## How we built it

We used Cohere classify for sentiment analysis and to filter out songs whose mood doesn't match the user's current state. Then we use Cohere's chat and RAG based on the database of filtered songs to identify songs based on the user prompt. We match images to music by first generating a caption of the images using the Azure computer vision API doing a semantic search using KNN and Cohere embeddings and then use Cohere rerank to smooth out the final choices. Finally we make the image come to life by generating a skeleton of the dance moves using OpenCV and Mediapipe and then using a pretrained model to transfer the skeleton to the image.

## Challenges we ran into

This was the most technical project any of us have ever done and we had to overcome huge learning curves. A lot of us were not familiar with some of Cohere's features such as re rank, RAG and embeddings. In addition, generating the skeleton turned out to be very difficult. Apart from simply generating a skeleton using the standard Mediapipe landmarks, we realized we had to customize which landmarks we are connecting to make it a suitable input for the pertained model. Lastly, understanding and being able to use the model was a huge challenge. We had to deal with issues such as dependency errors, lacking a GPU, fixing import statements, deprecated packages.

## Accomplishments that we're proud of

We are incredibly proud of being able to get a very ambitious project done. While it was already difficult to get a skeleton of the dance moves, manipulating the coordinates to fit our pre trained model's specifications was very challenging. Lastly, the amount of experimentation and determination to find a working model that could successfully take in a skeleton and output an "alive" image.

## What we learned

We learned about using media pipe and manipulating a graph of coordinates depending on the output we need, We also learned how to use pre trained weights and run models from open source code. Lastly, we learned about various new Cohere features such as RAG and re rank.

## What's next for retro.act

Expand our database of songs and dance videos to allow for more user options, and get a more accurate algorithm for indexing to classify iterate over/classify the data from the db. We also hope to make the skeleton's motions more smooth for more realistic images. Lastly, this is very ambitious, but we hope to make our own model to transfer skeletons to images instead of using a pretrained one.