## Inspiration 🔥

While on the way to CalHacks, we drove past a fire in Oakland Hills that had started just a few hours prior, meters away from I-580. Over the weekend, the fire quickly spread and ended up burning an area of 15 acres, damaging 2 homes and prompting 500 households to evacuate. This served as a harsh reminder that wildfires can and will start anywhere as long as few environmental conditions are met, and can have devastating effects on lives, property, and the environment.

*The following statistics are from the year 2020[1].*

**People:** Wildfires killed over 30 people in our home state of California. The pollution is set to shave off a year of life expectancy of CA residents in our most polluted counties if the trend continues.

**Property:** We sustained $19b in economic losses due to property damage.

**Environment:** Wildfires have made a significant impact on climate change. It was estimated that the smoke from CA wildfires made up 30% of the state’s greenhouse gas emissions. UChicago also found that “a single year of wildfire emissions is close to double emissions reductions achieved over 16 years.”

Right now (as of 10/20, 9:00AM): According to Cal Fire, there are 7 active wildfires that have scorched a total of approx. 120,000 acres.

[[1] - news.chicago.edu](https://news.uchicago.edu/story/wildfires-are-erasing-californias-climate-gains-research-shows)

## Our Solution: Canary 🐦🚨

Canary is an early wildfire detection system powered by an extensible, low-power, low-cost, low-maintenance sensor network solution. Each sensor in the network is placed in strategic locations in remote forest areas and records environmental data such as temperature and air quality, both of which can be used to detect fires. This data is forwarded through a WiFi link to a centrally-located satellite gateway computer. The gateway computer leverages a Monogoto Satellite NTN (graciously provided by Skylo) and receives all of the incoming sensor data from its local network, which is then relayed to a geostationary satellite. Back on Earth, we have a ground station dashboard that would be used by forest rangers and fire departments that receives the real-time sensor feed. Based on the locations and density of the sensors, we can effectively detect and localize a fire before it gets out of control.

## What Sets Canary Apart 💡

Current satellite-based solutions include Google’s FireSat and NASA’s GOES satellite network. These systems rely on high-quality **imagery** to localize the fires, quite literally a ‘top-down’ approach. Google claims it can detect a fire the size of a classroom and notify emergency services in 20 minutes on average, while GOES reports a latency of 3 hours or more. We believe these existing solutions are not effective enough to prevent the disasters that constantly disrupt the lives of California residents as the fires get too big or the latency is too high before we are able to do anything about it. To address these concerns, we propose our ‘bottom-up’ approach, where we can deploy sensor networks on a single forest or area level and then extend them with more sensors and gateway computers as needed.

## Technology Details 🖥️

Each node in the network is equipped with an Arduino 101 that reads from a Grove temperature sensor. This is wired to an ESP8266 that has a WiFi module to forward the sensor data to the central gateway computer wirelessly. The gateway computer, using the Monogoto board, relays all of the sensor data to the geostationary satellite. On the ground, we have a UDP server running in Google Cloud that receives packets from the satellite and is hooked up to a Streamlit dashboard for data visualization.

## Challenges and Lessons 🗻

There were two main challenges to this project.

**Hardware limitations:** Our team as a whole is not very experienced with hardware, and setting everything up and getting the different components to talk to each other was difficult. We went through 3 Raspberry Pis, a couple Arduinos, different types of sensors, and even had to fashion our own voltage divider before arriving at the final product. Although it was disheartening at times to deal with these constant failures, knowing that we persevered and stepped out of our comfort zones is fulfilling.

**Satellite communications:** The communication proved to be tricky due to inconsistent timing between sending and receiving the packages. We went through various socket ids and ports to see if there were any patterns to the delays. Through our thorough documentation of steps taken, we were eventually able to recognize a pattern in when the packages were being sent and modify our code accordingly.

## What’s Next for Canary 🛰️

As we get access to better sensors and gain more experience working with hardware components (especially PCB design), the reliability of our systems will improve. We ran into a fair amount of obstacles with the Monogoto board in particular, but as it was announced as a development kit only a week ago, we have full faith that it will only get better in the future. Our vision is to see Canary used by park services and fire departments in the most remote areas of our beautiful forest landscapes in which our satellite-powered sensor network can overcome the limitations of cellular communication and existing fire detection solutions.
## The Problem 🤔

In the midst of an ecological crisis, one of Canada's largest environmental problems is being caused by one of its smallest critters - the dreaded Mountain Pine Beetle. Between 1990 and 2012, the Beetle ate its way through approximately 723 million cubic metres (53%) of all the merchantable pine in BC, and it's on the rise again today.

Technology provides an obvious solution to this problem - drones and satellites can be used to scope out large areas of forest without needing to send forest rangers out, saving timber managers time and money. The beetle can then be eradicated via targeted bursts of insecticide sprayed from drones. Yet, existing insecticide spraying drones are expensive ($18000+ CAD) and can only spray on a predefined route, which is laborious to find and program.

## The Solution 💡

Hence, we present an all in one system for combatting this epidemic, comprising:

* Convolutional Neural Network model for detecting standing dead pines from satellite imagery, to enable automatic targeting of regions suffering from the beetle.
* Drone attachment for available hobby drones (cheap and could be "borrowed" by national park service when not in use by local citizens) to enable spraying of trees on the periphery of an affected area with insecticide to create a "firebreak".
* Web server with 3D globe for viewing the drone's progress in eradicating the beetle

The satellite imagery dataset could easily be replaced with data collected from the drone camera itself when moving to production, and the model retrained in a few minutes, though we weren't able to create this dataset ourselves due to not living in Canada.

## How we built it 🤖

A [BBC micro:bit](https://en.wikipedia.org/wiki/Micro_Bit) is mounted to the drone and attached to a servo motor used to release the insecticide. The micro:bit uses standard 2.4GHz radio to communicate with another micro:bit on the ground, using protocol buffers as the communication medium. We use protocol buffers as they enable us to send a large amount of data efficiently, in the small amount of bytes available over the radio.

The second micro:bit uses serial to communicate with the computer running the web server that displays the status of the project. A WebGL Earth 3D globe with Mapbox satellite tiles is used to depict where the drone is and the area it needs to cover. We use an API request to feed this data into the JS, after the server decodes it from protobuf.

![A schematic diagram explaining the structure of the project](https://media.discordapp.net/attachments/1010468691567202394/1010601723699679354/unknown.png?width=1080&height=386 "Structure of the Project")

We tested several machine learning models on satellite imagery and found that a CNN was the most effective model.

## Challenges we ran into 😬

* Trying to get protobufs to work on the micro:bit which has only 16 KiB of usable RAM. To solve this problem we developed a custom protocol buffer implementation for just the fields we need to send to load onto the micro:bit, and interface with standard protocol buffers code running on the server computer.
	+ Learning about different architectures in Tensorflow and fine tuning them.
	+ Building a hardware project remotely with only one team member having access to the hardware, and getting effective drone footage for the video!

## Accomplishments that we're proud of 🏆

* Having the confidence to try a challenging project using technology we hadn't used before in a hackathon, given that we weren't sure it was going to work.
* Integrating protocol buffers into the micro:bit via a custom library!
* Achieving 89% accuracy on the binary classifier model used to detect standing dead trees.

## What we learned 📚

* Protocol Buffers serialisation format, use of proto files, use of `buf lint` and `protoc`.

The buf CLI was invaluable in ensuring that our proto schema was correct, efficient and designed with backwards compatibility in mind.
* How to collect a good dataset for machine learning and tune hyperparameters to get good results (full pipeline for a real world scenario, when you haven't been given a pre-built dataset).
* Controlling a servo motor with a micro:bit.

## What's next for Pine Protection 🔮

To be scaled up the project would need to be supported by forest owners. The model could easily be retrained to deal with drone photos to ensure that the precision is better than you would get from satellites, then the product would need to be mass-manufactured as a small PCB which could be added onto commercially available hobby drones.
## Inspiration

Whether there is a fire threatening the Notre Dame, wildfires invading California's forests, or a small city fire, emergency services have difficulty arriving at the location.

## What it does

When Phoenix is activated via the web app, the carbon dioxide sensor, or the heat sensor, it flies to the location and tracks the fire. At the same time, the data from the sensors and a live video is displayed on the web app so emergency services can monitor the fire. When Phoenix locates the fire, it sprays an extinguishing agent ('silly string' Pennapps's birthday theme :)) at the fire!

## How I built it

We created the web app with Python, JSON, and CSS, Google Cloud Vision and hacked the movement of the drone with the PS drone API. The silly string is fired using a micro servo controlled by an esp32!

## Challenges I ran into

-Since a laptop can only be connected to one wifi network at a time, we needed to find a wifi adapter
-integrate the parts of the project (ml, web app, drone movement, servo control)

## Accomplishments that I'm proud of

* We integrated all the parts of the project

## What I learned

-Python, PS Drone API, Google Cloud Vision detection

## What's next for Phoenix

We hope to be able to add more sensors, and use Phoenix for natural disaster relief too.