## Inspiration

The inspiration for *SurgeVue* came from the high risks associated with brain surgeries, where precision is critical and the margin for error is minimal. With a mortality rate that increases 15-fold for non-elective neurosurgeries, we were motivated to create a solution that leverages augmented reality (AR) and machine learning to enhance the precision of brain tumor removal surgeries. By merging real-time data with surgical tools, we aimed to provide surgeons with an advanced assistant to reduce risks and save lives.

## What it does

*SurgeVue* is an AR-powered surgical assistant that provides neurosurgeons with real-time visual overlays during brain surgery. Using machine learning, the system outlines tumors on the patient's brain and classifies the presence of foreign objects. Integrated with an Arduino gyroscope for hand tracking, SurgeVue offers surgeons real-time feedback on tool movements, sweat sensor data, and critical hand movement trends—all displayed within a secure mobile app that ensures patient data privacy using facial recognition and RFID technology. The system empowers surgeons to make more informed, precise decisions during the most delicate procedures.

## How we built it

We built *SurgeVue* using a combination of cutting-edge technologies:

* **OpenCV** for real-time tumor detection and hand movement detection for augmented view.
* **PyTorch** for classifying tumors.
* **Swift and SceneKit** to create an immersive AR environment that overlays tumor outlines onto the surgeon's view.
* **Arduino gyroscope** for tracking the surgeon's hand movements and tool positioning.
* **PropelAuth** to ensure secure access to sensitive patient data via facial recognition and RFID.
* **Flask backend** to process machine learning models and serve image classification results via API.
* **Mobile App** that visualizes gyroscope, sweat sensor, and hand movement trends.

## Challenges we ran into

One of the biggest challenges was ensuring that the AR overlay, tumor detection, and hand-tracking happened in real-time without latency. We had to optimize our models to ensure seamless performance in the fast-paced environment of an operating room. Integrating the hardware components like the Arduino gyroscope and managing precise hand-tracking also posed challenges, as did creating a user-friendly interface that was informative without being overwhelming during a surgery.

## Accomplishments that we're proud of

* Successfully implementing real-time AR overlays that provide surgeons with critical information at a glance.
* Developing a machine learning model that accurately classifies tumors and detects foreign objects.
* Integrating hardware sensors (gyroscope, sweat sensors) to provide surgeons with hand movement insights, enhancing precision during surgeries.
* Ensuring patient data security through advanced authentication measures like facial recognition and RFID.

## What we learned

We learned how to combine AR and machine learning into a cohesive solution that can operate in real-time under intense conditions like surgery. We also gained experience in integrating hardware components, optimizing machine learning models for low latency, and handling large datasets like medical imaging. Furthermore we can save OR nurses and surgeons from intense radiation from the Medtronic devices, ones that prevent them from continuing operation. Additionally, building an intuitive, non-intrusive interface for surgeons highlighted the importance of user-centered design in healthcare applications.

## What's next for SurgeVue

Next, we plan to:

* **Refine the Machine Learning Model**: Enhance tumor classification accuracy and expand it to detect other conditions and anomalies.
* **Clinical Trials**: Test *SurgeVue* in real-world surgical settings and gather feedback from neurosurgeons.
* **Tool Tracking**: Further refine the hand-tracking and integrate more advanced surgical tools into the AR environment.
* **Global Expansion**: Implement support for other AR platforms like Hololens and explore expanding the use of the system in other complex surgeries beyond neurosurgery.
* **3D Implementation**: Create 3D models of the brain for the surgeon to interact with in real-time
## Inspiration

Physiotherapy is expensive for what it provides you with, A therapist stepping you through simple exercises and giving feedback and evaluation? WE CAN TOTALLY AUTOMATE THAT! We are undergoing the 4th industrial revolution and technology exists to help people in need of medical aid despite not having the time and money to see a real therapist every week.

## What it does

IMU and muscle sensors strapped onto the arm accurately track the state of the patient's arm as they are performing simple arm exercises for recovery. A 3d interactive GUI is set up to direct patients to move their arm from one location to another by performing localization using IMU data. A classifier is run on this variable-length data stream to determine the status of the patient and how well the patient is recovering. This whole process can be initialized with the touch of a button on your very own mobile application.

## How WE built it

on the embedded system side of things, we used a single raspberry pi for all the sensor processing. The Pi is in charge of interfacing with the IMU while another Arduino interfaces with the other IMU and a muscle sensor. The Arduino then relays this info over a bridged connection to a central processing device where it displays the 3D interactive GUI and processes the ML data. all the data in the backend is relayed and managed using ROS. This data is then uploaded to firebase where the information is saved on the cloud and can be accessed anytime by a smartphone. The firebase also handles plotting data to give accurate numerical feedback of the many values orientation trajectory, and improvement over time.

## Challenges WE ran into

hooking up 2 IMU to the same rpy is very difficult. We attempted to create a multiplexer system with little luck.
To run the second IMU we had to hook it up to the Arduino. Setting up the library was also difficult.
Another challenge we ran into was creating training data that was general enough and creating a preprocessing script that was able to overcome the variable size input data issue.
The last one was setting up a firebase connection with the app that supported the high data volume that we were able to send over and to create a graphing mechanism that is meaningful.
## Inspiration

We're interested in data and modelling it. The EDR challenge was a chance for us to learn about graph databases and their capabilities.

## What it does

A pipeline to process address data and perform some form of resolution. Constructs a graph DB representation in neo4j that contains hierarchical location data and information about purposes of places. The DB engine then allows us to construct queries such as searching for nearby places or looking at movements of address listings over time.

## How we built it

We used python and scipy to process the data. First, it sanitizes and performs address-resolution via a rule-based approach. Next, it polls longitude and latitude data from the Google Maps API. To support fuzzy location based searches, the points were converted to a suitable cartesian projection and the Delaunay triangulation is computed. The edges from the planar graph are used to support the fuzzy location based searches.

## Challenges we ran into

The address dataset is pretty big and took considerable work to clean up. We also had to keep efficiency in mind, which is why we computed the Delaunay triangulation, which only constructs O(n) edges.

## Accomplishments that we're proud of

* Learning what neo4j is capable of.
* Loading the data into a graph database in a way that supports non-trivial queries.
* Creating a workable search method for nearby places with large graphs

## What we learned

* What neo4j is capable of.
* Use of cloud-based geocoding capability for large datasets
* Benefits of a graph network to support fuzy search queries

## What's next for Places DB

More interesting queries can be explored, even with the graph structure that we currently have. The relationships between the user and venue could be tagged, like for shops or schools .etc.