## Inspiration

Our team is all out of state or international, and receiving health care here in California can be difficult. If receiving treatment away from your local hospital, the doctors may not have access to past history or current treatment plan. This can be a cost burden due to repeat diagnostic tests as well as a annoyance for both the patient and the provider. The healthcare workers should be able to use as much information as the patient is willing to give in order to make the best treatment plan.

## What it does

The treatment tracker will save records and can be organized to have individual trackers per condition and can record symptoms to bring into the doctor's visits. The doctors would have verified accounts to certify their diagnosis and lab work interpretations. It also can be used as a reminder for the patient regarding what they need to do daily to better their condition. It can also provide a timeline to show progress and encourage the patient to stick to the treatment plan.

## How we built it

We built it using MATLAB, and imported some design concepts such as the timeline from CANVA.

## Challenges we ran into

The biggest road block was our limited knowledge, we spent most of our time learning how to do things which left us short on time to implement everything. We also would have liked to add some front end and UX elements but we were very confused especially since none of us know any java.

## Accomplishments that we're proud of

We are proud that we made something! We are also proud of our problem because it is widespread yet unsolved.

## What we learned

We learned java, HTML, and CSS (the very basics) and decided that it was an up hill battle for this hackathon. We learned how to create an app in MATLAB which none of us knew was possible.

## What's next for Treatment Tracker

We plan to finish the Treatment Tracker and continue to learn java as a team so that we can enter another hackathon and have a much nicer looking program.
## Inspiration

In response to the challenges faced by the elderly and disabled who rely on traditional, costly service dogs, our team drew inspiration to create SpotAssist. Traditional service dogs demand substantial resources, extensive training, and are limited to one user at a time. SpotAssist leverages Boston Dynamics’ Spot to overcome these challenges, offering advanced functionalities through technologies like machine learning, computer vision, and natural language processing. This innovative robotic companion not only surpasses traditional service dogs but also holds the potential for mass deployment in settings like senior care homes, addressing financial and logistical constraints. SpotAssist embodies our commitment to making a significant impact by combining technology, compassion, and scalability to enhance the lives of those in need.

## What it does / How we built it

SpotAssist is an assistive robot meant to achieve *better* functionality than a regular service dog.
First, Spot will take an image of its surroundings and generate a list of objects detected along with their corresponding bounding boxes (using YOLO). Then, Spot will wait for the user to make a verbal query including a target landmark to approach; an example of such a command would be “Take me to the nearest chair.” Finally, Spot will gently guide the user to their desired destination through noting the difference between the center x-coordinate of the bounding box and the actual center of the image. Spot will utilize this quantity in order to align its heading angle to be in line with the desired object and subsequently move forward, stopping at a reasonable distance from the object in question. Furthermore, we built health-first integrations from the ground-up for Spot. Using our custom ML and analytical methods and Terra API integrations, we are able to use Spot as an alert for a medical event, such as abnormal heart rhythms and/or heart rate.

## Challenges we ran into

There were a number of technical challenges we encountered throughout the development process.
Firstly, a major hurdle was the restrictions placed on the compute module attached to Spot; we were not allowed to perform any commands directly to Spot itself (we instead had to communicate through the module). Furthermore, we were not allowed to SSH into the module (nor were we given elevated privileges), so we had to build our own version of a remote communication system between our remote servers and Spot. This led to many creative avenues being explored for development.

## Accomplishments that we're proud of

We’re especially proud of building upon a system/environment that was entirely new to the whole team, especially without any documentation on the compute module we built upon. Furthermore, none of the team knew each other prior to the hackathon, so we’re proud of the teamwork skills we were able to utilize to accomplish our success this weekend.

## What's next for SpotAssist

Mobile integration!
## Inspiration

Textbooks have not fundamentally changed since their invention in the 16th century. Although there are now digital textbooks (ePubs and the like), they're still just pictures and text. From educational literature, we know that discussion and interactivity is crucial for improving student outcomes (and, particularly, those of marginalized students). But we still do the majority of our learning with static words and images on a page.

## What it does

How do we keep students engaged? Introducing *Talk To History*. This is a living textbook, where students can read about a historical figure, click on their face on the side of the page, and have an immersive conversation with them. Or, read a primary text and then directly engage to ask questions about the writing. This enables a richer, multimodal interaction. It makes history more immersive. It creates more places to engage and retain knowledge. Most importantly, it makes the textbook fun. If Civ5 can make history fun, why can’t textbooks?

## How we built it

*Talk To History* was built using TypeScript, React, Next.js, Vercel, Chakra, Python, Google Text-To-Speech, Wav2Lip, GPT, and lots of caffeine :) The platform has several components, including a frontend for students and a backend to handle user data and text analysis. We also used Google's Text-To-Speech (TTS) API to generate high-quality speech output, which we then fed into Wav2Lip, a deep generative adversarial network, to produce realistic lip movements for the characters. For accelerated inference, we deployed Wav2Lip on an NVIDIA A40 GPU server.

## Challenges we ran into

* Dealing with CUDA memory leaks when performing inference using the Wav2Lip model
* Finetuning hyperparameters of the Wav2Lip model and optimizing PyTorch loading to reduce latency
* Connecting and deploying all of the different services (TTS, GPT, Wav2Lip) into a unified product

## Accomplishments we're proud of

We're most proud of building a platform that makes learning fun and engaging for students. On the technical side, we're proud of seamlessly integrating several cutting-edge technologies, such as Wav2Lip and GPT, to create a more immersive experience; this project required advanced techniques in full-stack engineering, multi-processing, and latency optimization. The end result was more than worth the effort, as we successfully created a platform that makes education more engaging and immersive. With *Talk To History*, we hope to transform the way students learn history.

## What we learned

We learned how to integrate multiple services and optimize our code to handle large amounts of data, but perhaps more importantly, we gained a deep appreciation for the importance of creating an exciting experience for students.

## What's next

* Scalability and speed improvements for Wav2Lip GPU instances for more realtime chats
* Improved robustness against adversarial prompts
* Broader selection of articles and speakers organized into different domains, such as "Pioneers in Environmental Sustainability", "Female Heroes in Science", and "Diverse Voices in Literature"
* *Talk to History* as a platform: ability for any educational content author to add their own character (subject to content approval) given some context and voice and integrate it on their website or e-reader