## Inspiration

Our project is inspired by the sister of one our creators, Joseph Ntaimo. Joseph often needs to help locate wheelchair accessible entrances to accommodate her, but they can be hard to find when buildings have multiple entrances. Therefore, we created our app as an innovative piece of assistive tech to improve accessibility across the campus.

## What it does

The user can find wheelchair accessible entrances with ease and get directions on where to find them.

## How we built it

We started off using MIT’s Accessible Routes interactive map to see where the wheelchair friendly entrances were located at MIT. We then inspected the JavaScript code running behind the map to find the latitude and longitude coordinates for each of the wheelchair locations.

We then created a Python script that filtered out the latitude and longitude values, ignoring the other syntax from the coordinate data, and stored the values in separate text files.

We tested whether our method would work in Python first, because it is the language we are most familiar with, by using string concatenation to add the proper Java syntax to the latitude and longitude points. Then we printed all of the points to the terminal and imported them into Android Studio.

After being certain that the method would work, we uploaded these files into the raw folder in Android Studio and wrote code in Java that would iterate through both of the latitude/longitude lists simultaneously and plot them onto the map.

The next step was learning how to change the color and image associated with each marker, which was very time intensive, but led us to having our custom logo for each of the markers.

Separately, we designed elements of the app in Adobe Illustrator and imported logos and button designs into Android Studio. Then, through trial and error (and YouTube videos), we figured out how to make buttons link to different pages, so we could have both a FAQ page and the map.

Then we combined both of the apps together atop of the original maps directory and ironed out the errors so that the pages would display properly.

## Challenges we ran into/Accomplishments

We had a lot more ideas than we were able to implement. Stripping our app to basic, reasonable features was something we had to tackle in the beginning, but it kept changing as we discovered the limitations of our project throughout the 24 hours. Therefore, we had to sacrifice features that we would otherwise have loved to add.

A big difficulty for our team was combining our different elements into a cohesive project. Since our team split up the usage of Android Studio, Adobe illustrator, and programming using the Google Maps API, it was most difficult to integrate all our work together.

We are proud of how effectively we were able to split up our team’s roles based on everyone’s unique skills. In this way, we were able to be maximally productive and play to our strengths.

We were also able to add Boston University accessible entrances in addition to MIT's, which proved that we could adopt this project for other schools and locations, not just MIT.

## What we learned

We used Android Studio for the first time to make apps. We discovered how much Google API had to offer, allowing us to make our map and include features such as instant directions to a location. This helped us realize that we should use our resources to their full capabilities.

## What's next for HandyMap

If given more time, we would have added many features such as accessibility for visually impaired students to help them find entrances, alerts for issues with accessing ramps and power doors, a community rating system of entrances, using machine learning and the community feature to auto-import maps that aren't interactive, and much, much more. Most important of all, we would apply it to all colleges and even anywhere in the world.
# Inspiration

Many cities in the United States are still severely behind on implementing infrastructure improvements to meet ADA (Americans with Disabilities Act) accessibility standards. Though 1 in 7 people in the US have a mobility-related disability, research has found that 65% of curb ramps and 48% of sidewalks are not accessible, and only 13% of state and local governments have transition plans for implementing improvements (Eisenberg et al, 2020). To make urban living accessible to all, cities need to upgrade their public infrastructure, starting with identifying areas that need the most improvement according to ADA guidelines. However, having city dispatchers travel and view every single area of a city is time consuming, expensive, and tedious. We aimed to utilize available data from Google Maps to streamline and automate the analysis of city areas for their compliance with ADA guidelines.

# What AcceCity does

AcceCity provides a machine learning-powered mapping platform that enables cities, urban planners, neighborhood associations, disability activists, and more to identify key areas to prioritize investment in. AcceCity identifies both problematic and up-to-standards spots and provides an interactive, dynamic map that enables on-demand regional mapping of accessibility concerns and improvements and street views of sites.

### Interactive dynamic map

AcceCity implements an interactive map, with city and satellite views, that enables on-demand mapping of accessibility concerns and improvements. Users can specify what regions they want to analyze, and a street view enables viewing of specific spots.

### Detailed accessibility concerns

AcceCity calculuates scores for each concern based on ADA standards in four categories: general accessibility, walkability, mobility, and parking. Examples of the features we used for each of these categories include the detection of ramps in front of raised entrances, the presence of sidewalks along roads, crosswalk markings at street intersections, and the number of handicap-reserved parking spots in parking lots. In addition, suggestions for possible solutions or improvements are provided for each concern.

### Accessibility scores

AcceCity auto-generates metrics for areas by computing regional scores (based on the scan area selected by the user) by category (general accessibility, walkability, mobility, and parking) in addition to an overall composite score.

# How we built it

### Frontend

We built the frontend using React with TailwindCSS for styling. The interactive dynamic map was implemented using the Google Maps API, and all map and site data are updated in real-time from Firebase using listeners.

New scan data are also instantly saved to the cloud for future reuse.

### Machine learning backend

First, we used the Google Maps API to send images of the street view to the backend. We looked for handicapped parking, sidewalks, disability ramps, and crosswalks and used computer vision, by custom-fitting a zero shot learning model called CLIP from OpenAI, to automatically detect those objects from the images. We tested the model using labeled data from Scale Rapid API.

After running this endpoint on all images in a region of interest, users can calculate a metric that represents the accessibility of that area to people with disabilities. We call that metric the ADA score, which can be good, average, or poor. (Regions with a poor ADA score should be specifically targeted by city planners to increase its accessibility.) We calculated this ADA score based on features such as the number of detected ramps, handicapped parking spaces, crosswalks, and sidewalks from the google maps image analysis discussed previously, in addition to using the number of accidents per year recorded in that area. We trained a proof of concept model using mage.ai, which provides an intuitive and high-level way to train custom models.

## Challenges we ran into

* Applying ML to diverse urban images, especially since it’s so “in the wild”
* Lack of general ML models for accessibility prediction
* Developing methods for calculating representative / accurate metrics
* Running ML model on laptops: very computationally expensive

## Accomplishments that we're proud of

* We developed the first framework that connects Google Maps images with computer vision models to analyze the cities we live in.
* We developed the first computer vision framework/model aimed to detect objects specific for people with disabilities
* We integrated the Google Maps API with a responsive frontend that allows users to view their areas of interest and enter street view to see the results of the model.

## What we learned

* We learned how to integrate the Google Maps API for different purposes.
* We learned how to customize the OpenAI zero shot learning for specific tasks.
* How to use Scale Rapid API to label images
* How to use Mage.ai to quickly and efficiently train classification models.

## What's next for AcceCity

* Integrating more external data (open city data): public buildings, city zoning, locations of social services, etc.
* Training the machine learning models with more data collected in tandem with city officials.

## Ethical considerations

As we develop technology made to enable and equalize the playing field for all people, it is important for us to benchmark our efforts against sustainable and ethical products. Accecity was developed with several ethical considerations in mind to address a potentially murky future at the intersection of everyday life (especially within our civilian infrastructure) and digital technology.

A primary lens we used to assist in our data collection and model training efforts was ensuring that we collected data points from a spectrum of different fields. We attempted to incorporate demographic, socioeconomic, and geopolitical diversity when developing our models to detect violations of the ADA. This is key, as studies have shown that ADA violations disproportionately affect socioeconomically disadvantaged groups, especially among Black and brown minorities.

By incorporating a diverse spectrum of information into our analysis, our outputs can also better serve the city and urban planners seeking to create more equitable access to cities for persons with disabilities and improve general walkability metrics.

At its core, AcceCity is meant to help urban planners design better cities. However, given the nature of our technology, it casts a wide, automatic net over certain regions. The voice of the end population is never heard, as all of our suggestion points are generated via Google Maps. In future iterations of our product, we would focus on implementing features that allow everyday civilians affected by ADA violations and lack of walkability to suggest changes to their cities or report concerns. People would have more trust in our product if they believe and see that it is truly creating a better city and neighborhood around them.

As we develop a technology that might revolutionize how cities approach urban planning and infrastructure budget, it is also important to consider how bad actors might aim to abuse our platform. The first and primary red flag is from the stance of someone who might abuse disability and reserved parking and actively seeks out those reserved spaces, when they have not applied for a disability placard, excluding those who need those spaces the most. Additionally, malicious actors might use the platform to scrape data on cities and general urban accessibility features and sell that data to firms that would want these kinds of metrics, which is why we firmly commit to securing our and never selling our data to third parties.

One final consideration for our product is its end goal: to help cities become more accessible for all. Once we achieve this goal, even on an individual concern by concern basis we should come back to cities and urban planners with information on the status of their improvements and more details on other places that they can attempt to create more equitable infrastructure.
## Inspiration ✨

Exploration is more than just being called Dora and staying away from Swiper. Yet the struggle to truly live out this experience continues to be a struggle for those with disabilities. We wanted to build a project that helps the visually impaired explore the world. We wanted to be their eyes (hence the play on words in our name).

## What it does 🤳

EyeExplore starts with a facial recognition login–a way for the visually impaired to log in without having to type out a password with the small letters on their phone. This information is stored in the Estuary database to be accessed in the future. It later brings them to a camera that detects surrounding obstructions with a text-to-speech function in order to read the names of these obstructions to know how to steer clear. This is done with the co:here API.

## How we built it 🔧

Building off many different API’s, there were many unique technologies and frameworks we had to build around. For the back end, we used Face++ API for facial recognition sign-in, Estuary/Firebase for facial recognition storage, co:here API for text-to-speech features, and the Local Python server for the backend. For the front end, we used Figma and React Native first designing and then implementing the designs into an Expo app.

## Challenges we ran into 🏃

We didn’t just run into challenges, the challenges chased us down. Whoever said it was a good idea for 4 first-years to build an almost functioning React Native app despite never having touched Native, must’ve been crazy (no one did by the way).

Halfway in, we realized the backend coders could not incorporate their code to React Native so we all had to scrap our old code and write new ones to be compatible with Native.

On the same note, we wanted to implement Tensorflow with Expo however they were incompatible and that resulted in our code not working out.

The Estuary database server also went down and we forgot to check the discord server for updates so we had to use firebase.

We struggled with “Possible unhandled promise rejection” and still don’t know how to solve this issue

JSON file syntax error which we did not find and that is what stopped us from getting further with our code. First, it was a broken promise but after fixing that error, we were unable to find the syntax error in all the JSON files.
We were also too hyper-fixated on individual problems so it was hard to move on from the code we took a long time to build.

## Accomplishments that we're proud of 🎉

Trying project ideas we have 0 experience in developing! Every hackathon we try to hack something new and time and time again we underestimate its difficulty. Halfway through this hackathon, we did lose a bit of hope, unsure if we were able to successfully develop the app by the deadline. However, we took a break, worked on our planning, and were able to voice out our concerns, resulting in us having something we were decently satisfied with (so close to full integration).

We are also very proud that we were able to get the app going in the first place as we, once again, had no experience with Native and we’re also new to fronted integration in general. To be able to even configure buttons, bars with input and output, and authentication is already crazy to us that we were able to figure it out.

## What we learned 🌱

We learned that while aiming high is good, we lose sight of feasibility. We were definitely in a time crunch toward the end, especially after pretty much restarting the project halfway through. We were also able to work on managing databases, more API implementation, and debugging (especially with Native).

## What's next for EyeExplore 🤩

Full integration of all parts. We all had the same clear vision for this project hence all parts of this project have been done by respective members. However, due to technical issues, we were unable to finish. We also hope to implement a translation feature for international users and a way to measure the distance between the person and the objects. The opportunities for EyeExplore are endless and we hope to finish our project in the near future!