*The tech is in your hands. [Now Scribble.](https://nowscribble.com)*

Graffiti has long been celebrated as the art of the people, but not all of us agree with risking law and limb to color the streets. At the same time, maturing Augmented Reality (AR) technologies have made it easier than ever for *anyone* to access a virtual world of infinite possibility through their smartphone (even very low-end ones).

**Scribble** is an experiment in wedding the tool to the need to create the next big trend in popular art: painting surfaces in the world with AR graffiti that persist forever for everyone to see. With no legal repercussions and infinite expressive value, we envision Scribble to be an opportunity to nurture creativity, foster communities, and enfranchise everyone through the power of ARt.

## The Tech

Scribble's core technologies are Apple's ARKit for AR capabilities and Wix Code as a heavy-lifting backend API and database.

Apple's ARKit provides a framework for iPhones that nicely abstracts-away the difficult computer-vision challenges required to build a 3D model of the world from a video stream. In particular, it allows the extraction of *Worlds, Anchors, and Scenes* from each AR session which can be serialized and stored in the cloud to later reproduce the same viewing experience for an arbitrary number of users.

We used Wix Code to implement this business-logic of storing and serving AR session assets. We initially decided to adopt it because it required no additional setup of a server instance (or the tedious process of setting up boilerplate code and installing packages). We were pleased to quickly observe that Wix Code's features were sophisticated enough for our needs, and even allowed us to implement our own token-based authentication middleware for validating access from the mobile app.

The mobile app using Apple's ARKit was coded in Swift and deployed for iOS mobile, and our Wix Code API and database logic was implemented in JavaScript (ES6) using a standard REST paradigm to interact with our database objects.

## Brick Walls

During our project, we ran into a number of brick walls which impeded our progress. But as any graffiti artist knows, such walls are just canvases which one can probe, freestyle, and use to grow.

* ARKit has many features that are technically sophisticated and, being rather new, are not well-documented. To figure out exactly what we needed to do to bend ARKit to our needs, we adopted a regime of enlightened trial-and-error which eventually led to our discovery of using Worlds, Anchors, and Scenes to persist a session.
* We were hoping to use Wix Code's `wix-users` or `wix-users-backend` libraries for user authentication, but after some experimentation and inquiry with the Wix mentors it became apparent that this was meant more as a frontend sugar than a Login API. After recognizing this, we were able to quickly pivot and implement our own authentication middleware using Wix Code's database and backend JS capabilities. We were even able to implement security measures (salted password hashes) by importing a SHA-256 library.
* During testing, we discovered a memory leak in the Wix Database Editing UI which caused the page to consume more than 5GB of RAM and crash in the course of displaying just a few rows containing 10s of MB of data. We suspect that this may have been our use of Text fields to store large amounts of base64-encoded serialized data; there might have been a better way to store this data. In the end, we were able to workaround this problem by running the webpage on a computer with more RAM.

## Masterpieces

If brick walls are canvases upon which we can shine, we think that the last 36 hours have lent us plenty of opportunities to be proud of what we've accomplished, including

* Successfully recreating AR sessions purely from independently saved data store in the cloud.
* Successfully implementing logic allowing users to build upon past sessions, enabling a collaborative art environment.
* Successfully implementing business logic for an API and database after using Wix for the very first time.
* Gaining sufficient comfort with Wix Code's features to implement a custom authentication middleware purely in Wix Code.
* Building something cool and having a lot of fun along the way!

## The Adventure

We've sure hit the streets with this Hackathon; and as street knowledge goes, we've learned a lot:

* Learning from our teammates about the Swift and the Apple development ecosystem after coming primarily from Android and React development.
* Discovering how to use ARKit through a hands-on, experimental approach.
* Learning about Wix's and Wix Code's many useful features for the first time (we were impressed by how easy it was to make our cool website at <https://nowscribble.com> in so little time!)
* Learning the best places to sleep in the SOCH after all the couches have been taken ;)

## Scribble on the Streets?

We, the Scribble Team, feel genuinely captivated by the possibilities presented by Scribble. We feel that our proof-of-concept app already captures the fun in being able to make *persistent* modifications to the world in AR; with just a bit of polishing, we think our app can bring real joy, expressive enfranchisement, and community bonding as people collaborate to create a rich world of virtual ARt. With the right amount of dedication, we think we can turn Scribble from a hackathon project into a real tool for social good.

Draw anything. Anywhere. Together.

Now Scribble.
## Inspiration

Accessible learning? Child-oriented? Making math…fun?

When we were brainstorming ideas, we kept coming back to the idea of learning in an accessible manner, regardless of a person's circumstance or what technology they have access to. We also realized that learning anything, especially Math, can be very challenging for young children, especially if they’ve never used a computer before. So, we thought “what if we combined those two ideas?”

That’s how we came up with mathimati.co, a program that combines these ideas to make learning Math for young children interactive, accessible, and most of all, fun!

## What it does

Mathemati.co, as the name suggests, is all about learning math! When you run the program, you’ll see a colourful, visually appealing interface perfect for young children. All of the numbers from 0-9 are perfectly laid out on the top row, with the operations here on the bottom row, and boxes for the numbers and equation signs here.

Your goal is to get the random number the program generates. And it’s pretty simple, you just move your hand, drop of the numbers, and get the equation here! The program will print a success message when you’re correct, and a failure message when you’re wrong

## How we built it

Mathematic.co was built in Python using OpenCV, Mediapipe and PyGame. Mediapipe was used for the hand tracking, allowing a point on the hand to be mapped to a point in the game. OpenCV was used for transferring the input from a camera to the computer itself. PyGame was used to create all of the objects in the game, such as the letters, boxes, equals sign, circles, all of that!

## Challenges we ran into

Building a fully fledged application in just 36 hours is very tough.

At our last hackathon, Deltahacks X, we chose to create a proof of concept since Adam was busy with clubs at McMaster that same day, but since we had more time for Deerhacks, we wanted to make it as perfect as possible. But with Adam and Gavin having to arrive late due to school and work, that still cut the time we had to design to just about 24 hours. We worked super hard for the entire time we had, but that isn’t a lot of time. We had so many ideas to expand on, but had to scrap a lot of them to fit within the time constraint.

We also had a whole lot of trouble splitting up the project which led to a huge time crunch right near the end.

## Accomplishments we’re proud of

I’m really proud of myself for being able to do the front end with PyGame! I’ve always wanted to learn PyGame, and simply creating the objects was a very fun experience for me! I was able to write the vast majority of code myself, and was really happy I was able to apply everything from high school computer science and my engineering course at McMaster to this project - Adam

## What we learned

Time management and proper planning is everything in a hackathon. We had so many ideas of what we could do, but we realized that, above all, it’s super important to narrow our focus to a manageable idea that can actually be done within a 36 hour time frame. As we mentioned it led to a huge time crunch near the end of the project as we were all super reliant on each other’s parts of the project.

Next time, it’s very important for us to split up the project in a way that’s more manageable, so we can get work done more efficiently…and actually get sleep too.

## What’s next

There’s so many ways our idea can be expanded upon! This includes…
Adding multiplication and division
Starting by only dragging one number instead of both
Adding two digit numbers
Creating a time limit so children have to solve as many questions as they can in a certain amount of time

All in all, this was a super fun project for us! Not only was it amazing to see an idea like this actually come to fruition, but it was amazing for us to all see each other again, with us all being at different universities today.

And with what we’ve created, the future is super bright when it comes to teaching Math to young children. By creating a program that’s interactive, accessible and fun, mathemati.co is perfect for helping everyone learn and excel at Math from a young age, all with just the wave of a hand!
Demo: <https://youtu.be/cTh3Q6a2OIM?t=2401>

## Inspiration

Fun Mobile AR Experiences such as Pokemon Go

## What it does

First, a single player hides a virtual penguin somewhere in the room. Then, the app creates hundreds of obstacles for the other players in AR. The player that finds the penguin first wins!

## How we built it

We used AWS and Node JS to create a server to handle realtime communication between all players. We also used Socket IO so that we could easily broadcast information to all players.

## Challenges we ran into

For the majority of the hackathon, we were aiming to use Apple's Multipeer Connectivity framework for realtime peer-to-peer communication. Although we wrote significant code using this framework, we had to switch to Socket IO due to connectivity issues.

Furthermore, shared AR experiences is a very new field with a lot of technical challenges, and it was very exciting to work through bugs in ensuring that all users have see similar obstacles throughout the room.

## Accomplishments that we're proud of

For two of us, it was our very first iOS application. We had never used Swift before, and we had a lot of fun learning to use xCode. For the entire team, we had never worked with AR or Apple's AR-Kit before.

We are proud we were able to make a fun and easy to use AR experience. We also were happy we were able to use Retro styling in our application

## What we learned

-Creating shared AR experiences is challenging but fun
-How to work with iOS's Multipeer framework
-How to use AR Kit

## What's next for ScavengAR

* Look out for an app store release soon!