### We are Ally. Our mission is to save lives by automating drone data management.

Continue reading to see the ethical analysis of our project!

## **Inspiration**

*Tens of thousands of people are dying every year in natural disasters due to poor infrastructure.*

Months after the devastating 2008 earthquake in Sichuan, China, Mindy visited the region and witnessed the horrific consequences of lacking up-to-code infrastructure. She spoke with locals, meeting a grandmother carrying a baby whose father was lost in the rubble when she was only 10 days old. Fast forward to today, the death tolls are now in the tens of thousands due to the Turkey-Syria earthquakes, with at least 20,000 victims still lying beneath the rubble. More than 6,000 buildings collapsed in the region, but a lot of the damage could have been prevented with better infrastructure monitoring.

These earthquakes were extremely deadly because these cities did not enforce building codes or treat safe housing as a human right (Vox News). Construction companies have been cutting corners and ignoring building codes for decades, and the government often just lets it slide.

Through researching current processes rescue teams use to help on-site, we started piecing together how impactful drones can be in this space. Specifically, we decided to create automated workflows because we’ve worked with drone data before and know that it’s a lot to take on as a first-time user.

## **What it does**

Ally makes drone data management simple, giving non-technical people easy access to automated workflows for their drone data. Commercial uses of drone technology have been most prevalent in fields such as agriculture, forestry management, urban planning, and disaster relief, where most users are not technically familiar with image processing. Our end-to-end platform allows users to drag-and-drop tasks (i.e. for our current focus of disaster management: locate people stranded under rubble; map 3D reconstructions of high-priority areas; annotate imagery and send to relevant teams) into a custom workflow. For example, a search-and-rescue official’s workflow could be [search(”people in rubble”), map/locate() results]. Our goal is to eliminate overhead and allow any users to efficiently comb through drone image data to make informed decisions.

During the hackathon, we focused on connecting drone images to semantic search, using OpenAI’s Clip neural network that brings together text and images. Users can upload drone images, search for keywords to resurface important timeframes, and sort images based on how close their query is.

## **What we learned**

* **Drones**: Though we’ve worked with drones before, Treehacks was the first time we had access to such high-tech drones from Skydio and Parrot. It was super inspiring to learn about how fast the industry is advancing and see how passionate their teams were about their technology.
* **Disaster Management**: We learned so much about the problems surrounding disaster management and narrowed down one of the major pain points– bad infrastructure. It was disappointing to hear that many of the lives lost in these natural disasters were very preventable if all parties acted ethically from the start.
* **OpenAI CLIP Model**: Most models like ChatGPT use only one form of input: text. OpenAI’s CLIP model is one of the first to effectively connect images and text by embedding them in the same vector space. This allows us to basically map out not only how close images are to each other in meaning, but also how close they are to words and phrases. Treehacks was the first time we’ve worked with an existing ML model with pre-trained weights—setting up was definitely a challenge.
* **Image search**: We learned that image search is hard to do. We considered alternative image search approaches, like using keywords, or other current AI methods. However, the CLIP model seemed to be the highest performing one. At the moment, CLIP performs well out-of-the-box without fine tuning. In the future, we see ways to improve the search by training CLIP further.

## **Future**

There are lots of interesting possibilities for the future. After we create the automated workflow platform, we can expand to different customer bases. Farmers can create workflows to monitor their land (identify areas with pests, monitor soil moisture, track movement and health of livestock) and alert relevant teams. City planners can create workflows to monitor urban growth, detect poor infrastructure, and report any damage. Construction workers can create workflows to send their drones on routine checkups, inspect infrastructure for potential problems, and send alerts to intervening teams.

### Business Model

The current plan is a freemium approach, where we give away free 1GB accounts and charge for # of users, additional storage, number of integrations, and/or computer power. We plan on first giving our platform for free out to disaster managers so they can setup their workflows and start using it for prevention methods. This way they are setup for success in the case that disaster strikes.

The global disaster management market is expected to grow to 5.2 billion by 2027. The global agricultural drone market is expected to reach USD 1.7 billion by 2025, and the global market for urban planning is expected to reach USD 7.6 billion by 2027. In addition, with the rise of extreme climate events, the need for disaster relief software and tools will only grow.

Automating workflows for the current existing manual processes in these fields will save immense time that is currently being used to do mundane, repetitive, logistical tasks, giving back time to spend on growth and innovation. Though money is important to run a business, we’re currently focused on saving critical time from disaster managers who will in turn be able to save more lives.

## **Ethics**

We strongly believe that governments should treat safe housing as a human right. After doing a lot of research into the space, we learned that construction companies have been cutting corners and ignoring building codes for decades, and the government often just lets it slide. This was one of the biggest reasons for the high death count after the recent earthquake in Turkey and Syria.

It’s ethically wrong for us as a society to allow people living in bad infrastructure to constantly be in worry about their livelihood. Or worse, not even realize how dangerous their living situation is. Thus, for this hackathon, we focused on safety because it was the most high-impact in saving lives.

The first step we took was building out the drone image search, giving disaster managers a quick way to sort through their large datasets. Not only will this be extremely useful post-disaster to filter out data, but also to help identify bad infrastructure areas and notify those in the area before it’s too late.

Ally makes drone data management simple and easily accessible for non-technical users. With such a high-impact space, there exist many ethical issues to further explore:

1. **Privacy** is an especially important consideration when using drones for data collection. The data collected by drones can be used to monitor people without their knowledge or consent, or to gain access to sensitive information. To ensure privacy, drones should only be used with explicit consent and with appropriate oversight or regulation. Additionally, all data collected should be treated with the utmost respect for the rights and privacy of the people involved.
2. **Accuracy** is also a key ethical concern when using drones for data collection. If the data and images collected are not accurate, then decisions made based on this data could be wrong or misguided. To ensure accuracy, the designers of the drones must ensure that the automated workflows and algorithms used are reliable and effective.
3. **Safety and security** are also important ethical considerations when using drones for data collection. If the data is used for disaster relief, for example, then there is the potential for drones to crash or malfunction, endangering the people in the area. Additionally, drone data is vulnerable to hacking, potentially leading to a breach of confidential information. To ensure safety and security, designers must ensure that the drones are equipped with appropriate safeguards and that the data collected is encrypted.
4. **Reliability** is a major ethical consideration when using drones for data collection. If the automated workflows are not reliable, then the data collected could be inaccurate and ineffective for decision making. To ensure reliability, designers must ensure that the drones are equipped with reliable and accurate sensors and that the data is stored securely.

By taking these ethical considerations into account when designing our product, Ally, we will ensure that the data is used responsibly and in the best interest of the public.

### Resources

* [Applications of drone in disaster management: A scoping review](https://www.sciencedirect.com/science/article/pii/S1355030621001477)
* [Vox News: How these buildings made the Turkey and Syria earthquakes so deadly](https://www.vox.com/videos/2023/2/16/23602986/turkey-syria-earthquake-soft-story-buildings-collapse)
## Inspiration and What it does

We often go out with a lot of amazing friends for trips, restaurants, tourism, weekend expedition and what not. Every encounter has an associated messenger groupchat. We want a way to split money which is better than to discuss on the groupchat, ask people their public key/usernames and pay on a different platform. We've integrated them so that we can do transactions and chat at a single place.
We (our team) believe that **"The Future of money is Digital Currency "** (-Bill Gates), and so, we've integrated payment with Algorand's AlgoCoins with the chat. To make the process as simple as possible without being less robust, we extract payment information out of text as well as voice messages.

## How I built it

We used Google Cloud NLP and IBM Watson Natural Language Understanding apis to extract the relevant information. Voice messages are first converted to text using Rev.ai speech-to-text. We complete the payment using the blockchain set up with Algorand API. All scripts and database will be hosted on the AWS server

## Challenges I ran into

It turned out to be unexpectedly hard to accurately find out the payer and payee. Dealing with the blockchain part was a great learning experience.

## Accomplishments that I'm proud of

that we were able to make it work in less than 24 hours

## What I learned

A lot of different APIs

## What's next for Mess-Blockchain-enger

Different kinds of currencies, more messaging platforms
**Introducing Bin Buddy, your intelligent waste-sorting companion!**

## Inspiration

In a world increasingly focused on **sustainability**, we saw a growing need for a tool that simplifies waste disposal. Inspired by the challenges people face when trying to recycle properly and reduce waste, we set out to create an innovative solution that would empower users to make informed choices.

## What It Does

Bin Buddy leverages machine learning with **TensorFlow** to identify whether an item is recyclable, compostable, or trash—all by simply taking a picture. Our user-friendly app provides real-time guidance, ensuring that you always know the right bin to use, helping to **reduce contamination** and promote responsible waste disposal.

## How We Built It

We built Bin Buddy using **React Native** for a seamless mobile experience, combined with **Expo** for cross-platform compatibility. The core waste classification is powered by TensorFlow, allowing for fast and accurate predictions directly on your device. The intuitive design utilizes **Tailwind CSS** to create a clean and responsive interface. And finally **Firebase** for robust authentication and real-time data processing with **Firestore**.

## Challenges We Ran Into

During development, we faced challenges training our machine learning model to accurately recognize a diverse range of waste items. Ensuring privacy and performance for on-device processing also required extensive testing and optimization. However, we implemented a clever optimization by mapping possible objects to their classifications, which not only reduced the necessary compute power but also improved the accuracy of our results.

## Accomplishments That We’re Proud Of

We are proud to have developed a reliable waste classification model capable of accurately identifying multiple waste types. Launching a cross-platform app that is both intuitive and engaging has been a significant milestone for our team. Remarkably, we accomplished this despite starting with zero experience in the technologies we used. The journey of learning and implementing an application with such potential has been quite a feat! Additionally, we’re thrilled to have fostered a community of eco-conscious users who are committed to making a difference.

## What We Learned

Throughout this journey, we learned the importance of user feedback in shaping our app’s features and interface. We also gained valuable insights into optimizing machine learning models for real-world applications and the significance of building an inclusive design that resonates with a broad audience.

## What’s Next for Bin Buddy

Looking ahead, we plan to expand Bin Buddy's capabilities by increasing our waste database, incorporating geolocation features for tailored disposal advice, and introducing gamification elements to motivate users. We also aim to offer multi-language support to reach a wider audience and foster global participation in sustainable practices.

Join us in our mission to make the world a cleaner, greener place—one waste item at a time!