## Inspiration

The present disconnect between an equation written on the board and the visual representation of that equation. As previous Multivariable Calculus students, we struggled to grasp the complex equations thrown at us that were not trivial to conceptualize.

## What it does

This application is that connection. We interpret handwritten math equations using OCR within Wolfram's Mathematica platform. This data is then used to generate visual 3d models which are layered over a chosen surface within augmented reality. Essentially you strap on your Hololens start the HoloWolfram app then scan a math equation to instantly see the 3d model right on your desk.

## How we built it

HoloWolfram was built using the Wolfram Mathematica platform to parse the data obtained from a picture taken with the Hololens Once Wolfram was able to successfully read the equations we were writing out by hand on a standard sheet of paper it then generates a 3d model of the equation. This 3d model is loaded into unity as a GameObject using the Unity Link beta that was graciously provided to us by Kyle from Wolfram. Once we were able to load the 3d model into Unity it was just a matter of bringing all these separate pieces together and building the project to the Hololens.

## Challenges we ran into

Each different piece had its own pitfalls and being new to all of the different technologies used in HoloWolfram's creation was not an advantage. Building a Minimum Viable Product to Hololens successfully was perhaps our greatest feat because there are just so many moving parts of an app such as this and under the strict time constraints of a hackathon, decisions have to be made swiftly.

## Accomplishments that we're proud of

-Successfully Using the Unity Link in its beta form
-Learning how to build applications for the Hololens
-Diving deep into the Unity development environment

## What we learned

-Wolfram Mathematica is extremely powerful.

## What's next for HoloWolfram

-More interactivity,
-Gamification,
-Polished UI, and
-Publication to the Windows Store
## What it does

ColoVR is a virtual experience allowing users to modify their environment with colors and models. It's a relaxing, low poly experience that is the first of its kind to bring an accessible 3D experience to the Google Cardboard platform. It's a great way to become a creative in 3D without dropping a ton of money on VR hardware.

## How we built it

We used Google Daydream and extended off of the demo scene to our liking. We used Unity and C# to create the scene, and do all the user interactions. We did painting and dragging objects around the scene using a common graphics technique called Ray Tracing. We colored the scene by changing the vertex color. We also created a palatte that allowed you to change tools, colors, and insert meshes. We hand modeled the low poly meshes in Maya.

## Challenges we ran into

The main challenge was finding a proper mechanism for coloring the geometry in the scene. We first tried to paint the mesh face by face, but we had no way of accessing the entire face from the way the Unity mesh was set up. We then looked into an asset that would help us paint directly on top of the texture, but it ended up being too slow. Textures tend to render too slowly, so we ended up doing it with a vertex shader that would interpolate between vertex colors, allowing real time painting of meshes. We implemented it so that we changed all the vertices that belong to a fragment of a face. The vertex shader was the fastest way that we could render real time painting, and emulate the painting of a triangulated face. Our second biggest challenge was using ray tracing to find the object we were interacting with. We had to navigate the Unity API and get acquainted with their Physics raytracer and mesh colliders to properly implement all interaction with the cursor. Our third challenge was making use of the Controller that Google Daydream supplied us with - it was great and very sandboxed, but we were almost limited in terms of functionality. We had to find a way to be able to change all colors, insert different meshes, and interact with the objects with only two available buttons.

## Accomplishments that we're proud of

We're proud of the fact that we were able to get a clean, working project that was exactly what we pictured. People seemed to really enjoy the concept and interaction.

## What we learned

How to optimize for a certain platform - in terms of UI, geometry, textures and interaction.

## What's next for ColoVR

Hats, interactive collaborative space for multiple users. We want to be able to host the scene and make it accessible to multiple users at once, and store the state of the scene (maybe even turn it into an infinite world) where users can explore what past users have done and see the changes other users make in real time.
## Inspiration

A week or so ago, Nyle DiMarco, the model/actor/deaf activist, visited my school and enlightened our students about how his experience as a deaf person was in shows like Dancing with the Stars (where he danced without hearing the music) and America's Next Top Model. Many people on those sets and in the cast could not use American Sign Language (ASL) to communicate with him, and so he had no idea what was going on at times.

## What it does

SpeakAR listens to speech around the user and translates it into a language the user can understand. Especially for deaf users, the visual ASL translations are helpful because that is often their native language.

![Image of ASL](https://res.cloudinary.com/devpost/image/fetch/s--wWJOXt4_--/c_limit,f_auto,fl_lossy,q_auto:eco,w_900/http://az616578.vo.msecnd.net/files/2016/04/17/6359646757437353841666149658_asl.png)

## How we built it

We utilized Unity's built-in dictation recognizer API to convert spoken word into text. Then, using a combination of gifs and 3D-modeled hands, we translated that text into ASL. The scripts were written in C#.

## Challenges we ran into

The Microsoft HoloLens requires very specific development tools, and because we had never used the laptops we brought to develop for it before, we had to start setting everything up from scratch. One of our laptops could not "Build" properly at all, so all four of us were stuck using only one laptop. Our original idea of implementing facial recognition could not work due to technical challenges, so we actually had to start over with a **completely new** idea at 5:30PM on Saturday night. The time constraint as well as the technical restrictions on Unity3D reduced the number of features/quality of features we could include in the app.

## Accomplishments that we're proud of

This was our first time developing with the HoloLens at a hackathon. We were completely new to using GIFs in Unity and displaying it in the Hololens. We are proud of overcoming our technical challenges and adapting the idea to suit the technology.

## What we learned

Make sure you have a laptop (with the proper software installed) that's compatible with the device you want to build for.

## What's next for SpeakAR

In the future, we hope to implement reverse communication to enable those fluent in ASL to sign in front of the HoloLens and automatically translate their movements into speech. We also want to include foreign language translation so it provides more value to **deaf and hearing** users. The overall quality and speed of translations can also be improved greatly.