## Inspiration

Although each of us came from different backgrounds, we each share similar experiences/challenges during our high school years: it was extremely hard to visualize difficult concepts, much less understand the the various complex interactions. This was most prominent in chemistry, where 3D molecular models were simply nonexistent, and 2D visualizations only served to increase confusion. Sometimes, teachers would use a combination of Styrofoam balls, toothpicks and pens to attempt to demonstrate, yet despite their efforts, there was very little effect. Thus, we decided to make an application which facilitates student comprehension by allowing them to take a picture of troubling text/images and get an interactive 3D augmented reality model.

## What it does

The app is split between two interfaces: one for text visualization, and another for diagram visualization. The app is currently functional solely with Chemistry, but can easily be expanded to other subjects as well.

If the text visualization is chosen, an in-built camera pops up and allows the user to take a picture of the body of text. We used Google's ML-Kit to parse the text on the image into a string, and ran a NLP algorithm (Rapid Automatic Keyword Extraction) to generate a comprehensive flashcard list. Users can click on each flashcard to see an interactive 3D model of the element, zooming and rotating it so it can be seen from every angle. If more information is desired, a Wikipedia tab can be pulled up by swiping upwards.

If diagram visualization is chosen, the camera remains perpetually on for the user to focus on a specific diagram. An augmented reality model will float above the corresponding diagrams, which can be clicked on for further enlargement and interaction.

## How we built it

Android Studio, Unity, Blender, Google ML-Kit

## Challenges we ran into

Developing and integrating 3D Models into the corresponding environments.
Merging the Unity and Android Studio mobile applications into a single cohesive interface.

## What's next for Stud\_Vision

The next step of our mobile application is increasing the database of 3D Models to include a wider variety of keywords. We also aim to be able to integrate with other core scholastic subjects, such as History and Math.
## Our Inspiration

We were inspired by apps like Duolingo and Quizlet for language learning, and wanted to extend those experiences to a VR environment. The goal was to gameify the entire learning experience and make it immersive all while providing users with the resources to dig deeper into concepts.

## What it does

EduSphere is an interactive AR/VR language learning VisionOS application designed for the new Apple Vision Pro. It contains three fully developed features: a 3D popup game, a multi-lingual chatbot, and an immersive learning environment. It leverages the visually compelling and intuitive nature of the VisionOS system to target three of the most crucial language learning styles: visual, kinesthetic, and literacy - allowing users to truly learn at their own comfort. We believe the immersive environment will make language learning even more memorable and enjoyable.

## How we built it

We built the VisionOS app using the Beta development kit for the Apple Vision Pro. The front-end and AR/VR components were made using Swift, SwiftUI, Alamofire, RealityKit, and concurrent MVVM design architecture. 3D Models were converted through Reality Converter as .usdz files for AR modelling. We stored these files on the Google Cloud Bucket Storage, with their corresponding metadata on CockroachDB. We used a microservice architecture for the backend, creating various scripts involving Python, Flask, SQL, and Cohere. To control the Apple Vision Pro simulator, we linked a Nintendo Switch controller for interaction in 3D space.

## Challenges we ran into

Learning to build for the VisionOS was challenging mainly due to the lack of documentation and libraries available. We faced various problems with 3D Modelling, colour rendering, and databases, as it was difficult to navigate this new space without references or sources to fall back on. We had to build many things from scratch while discovering the limitations within the Beta development environment. Debugging certain issues also proved to be a challenge. We also really wanted to try using eye tracking or hand gesturing technologies, but unfortunately, Apple hasn't released these yet without a physical Vision Pro. We would be happy to try out these cool features in the future, and we're definitely excited about what's to come in AR/VR!

## Accomplishments that we're proud of

We're really proud that we were able to get a functional app working on the VisionOS, especially since this was our first time working with the platform. The use of multiple APIs and 3D modelling tools was also the amalgamation of all our interests and skillsets combined, which was really rewarding to see come to life.
## Inspiration

Current students struggle with gaining the most value in the shortest time, We learned about the best methods of consuming information - visual and summaries, but current teaching techniques lack these methods.

## What it does

Scan your writing using your phone which then converts it into a text file. Then extract the keywords from that file, look up most relevant google photos from that and brings it back to the phone to display the summary with the photo and the text below. Options of more specific summaries for the individual keywords will be provided as well. It can also read aloud the summary for the visually impaired in order to have better traction and user interaction.

## How I built it

We used unity to develop an augmented reality environment , we integrated machine learning model at the backend of that environment using rest API. We used the google cloud vision API to scan images for text and then using that text as our input to produce our text to speech , summarization and slideshow capabilities

## Challenges I ran into

1.)Difficulties in interaction/ creating communication channels between Unity applications and python machine learning models due to lack of existence of documentation and resources for the same
2.) integration of python models into REST APIs
3.) Difficulties in uploading our API into an online cloud based web hosting service ,due to its large size

## Accomplishments that I'm proud of

1.)Successfully integrating two different platforms together which end up linking two very powerful and vast fields(AR and machine learning)
2.)Creating a working app in 2 days which happens to be an unique product on the market

## What I learned

1.) we learnt how to form a data pipeline between Unity and python which supports AR and machine learning
2.) how to form realtime applications that run ML at thier backend
3.) how to effectively manage our time and make a product in 2 days as a team activity
4.) how to make our own APIs and how to work with them ,

## What's next for TextWiz

We currently only support english , we wish to support more languages in the future and we wish to be able to translate between those language to increase connectivity on a global level.
Our current machine learning model while being good still has room for improvement and we plan to increase the accuracy of our models in the future.