## Inspiration

As of today, over a third of the US population is over the age of 50 and this number is rapidly rising. One of the biggest issues facing today's elderly is an inability to understand and effectively utilize modern technology such as the Internet. As they did not grow up with these tools, many seniors find technology to be intimidating and become averse to its usage entirely. As a result, the elderly population suffers from an inhibited access to information and services, limiting their autonomy. This can be particularly impactful when seniors are unable to access important health-related information. For example, a senior may be experiencing chest pain and want to learn more about their condition. Simply searching "chest pain" on Google returns over three billion search results, many of which are long articles about general chest pain and the possible causes, symptoms, and treatment options. While this may be helpful and digestible for someone who has grown up with the Internet, for the elderly, such a wide scope of information may be daunting and difficult to parse through. There are so many different possible causes for chest pain and each has its own set of symptoms and conditions. It would be much more effective for them to include in their search query more specifics about their situation (ie, 'sharp upper chest pain lasting for three weeks'). We hope to help them leverage the Internet in the ways that they would like, but don't know how to.

## What it does

Rondo is an AI-powered search-engine tool that makes the Internet truly accessible for the elderly population. Rondo takes in a query input from the user like any other regular old search engine. However, based off the specificity of the query, Rondo will prompt the user to answer a specific, well defined multiple choice question related to the original query in order to generate a more specific, better defined search query with more tailored results. Along with the question prompting, Rondo also automatically summarizes the top ten search results from the current query in order to make the content of each link more digestible to the user, who can read the brief summaries and decide which link to click on. These two functionalities run in parallel with question prompting on the left half of the screen and the article summaries on the right hand side. This way, the user can see what the current search query returns and can click on an article they find to be appropriate at any time. If the search results are still too broad or not what they are looking for, they can continue along with the prompted question to generate a better search query. Once the query reaches a certain level of specificity, Rondo will stop asking follow up questions.

## How we built it

For the frontend, we used Reflex -- an open-source framework for building web applications in pure python.
For the backend, we leveraged Python along with the OpenAI API, constructing prompts to elicit relevant responses, and dynamically updating queries based on iterative user input.

## Challenges we ran into

* Fine tuning GPT-4 for optimal query updating and follow-up question generation required persistent experimentation and tactical prompt engineering.
* Balancing the efficiency needs of a search engine with the processing time of our openai summarization tool.
* Integrating free-form GPT-4 responses into a highly structured frontend.

## Accomplishments that we're proud of

We are immensely proud of how much we were able to accomplish in such a short timeframe, especially given the context of our lack of front-end development experience. We were able to quickly pick up and leverage Reflex in order to create a fully functional product. We also put a lot of time and careful thought into our ideation process and we are very proud of the human-centered design we were able to create.

## What we learned

Through our fully operational implementation of Rondo, we gained a rich and thorough understanding of sophisticated language processing tools such as ChatGPT and more specifically how to adapt and fine-tune such powerful models for our specific applications. We also gained many insights into user experience design through our design process which aimed to create a user-friendly interface with the target audience of the elderly population in mind.

## What's next for Rondo - A Search Engine for the Elderly Made Easy

* Faster information retrieval and summary recall using more efficient LLMs and GPT prompts
* "Query Quality" bar to measure the specificity of the prompt and incentivize continued prompt specification
* More accessible summarization tools depending on the user's reading level

## Try it out:

* Clone and cd into the Git repository
* From the root directory, run `pip install -r requirements.txt`
* Replace the `secret_key` in `summarizeWebPage.py` and `openai.key` in `follow_up_question_generation.py` with your openai key
## Inspiration

The inspiration for Polaris came from witnessing the struggles that the elderly face in navigating the rapidly evolving digital landscape. In an age where the internet is integral to everyday life, it became clear that a significant portion of the population is left behind due to interfaces that are not designed with their needs in mind. Our goal was to create a solution that not only bridges this gap but also empowers the elderly to navigate the web with confidence and independence, ensuring that age is not a barrier to digital literacy and access.

## What It Does

Polaris is a Chrome extension that revolutionizes web accessibility for the elderly. By allowing users to describe in natural language what they wish to accomplish on a website, Polaris guides them through the necessary steps. It captures and analyzes web page components, interprets the user’s intent through advanced language models, and provides visual cues and simple instructions via an overlay. This process simplifies web navigation, making digital spaces more inclusive and user-friendly.

## How We Built It

We built Polaris using a combination of different frontend frameworks, APIs and machine learning technologies. The front end, developed as a Chrome extension and accompanying web app, captures user input and webpage elements processed using Beautiful Soup. The backend, powered by multi-modal language models, processes this data to understand the context and intent behind user commands. We leveraged open-source models like Llama2 and Mistral8x7B provided by together.ai's inference APIs and maximized their potential with prompt engineering. Our user-focused interface built with a combination of React, MaterialUI and Vite, allows us to simplify navigating the web for people of all age.

## Challenges We Ran Into

One of the main challenges was ensuring accurate segmentation and interpretation of web page components in real time, which required optimizing our models for speed without sacrificing accuracy. Another challenge was designing an intuitive user interface that could be easily navigated by elderly users, necessitating several iterations based on user feedback.

## Accomplishments That We're Proud Of

We are particularly proud of developing a solution that significantly improves web accessibility for the elderly, a demographic often overlooked in technology design. Successfully integrating complex AI technologies into a user-friendly application that can run efficiently as a Chrome extension stands as a testament to our team’s dedication and technical prowess.

## What We Learned

Throughout this project, we gained deeper insights into the challenges of web accessibility and the potential of AI to address these issues. We learned the importance of user-centered design, especially when creating technology for populations with specific needs. Additionally, we honed our skills in AI development, particularly in prompt engineering and data processing, and learned valuable lessons in teamwork, project management, and iterative design.

## What's Next for Polaris

Looking ahead, we aim to expand Polaris's capabilities to cover more complex web interactions and support additional browsers beyond Chrome. Our vision is a digital world whose benefits are universally accessible to everyone, irrespective of age. We see what is now just a simple but powerful chrome extension one day turning into a suite of tools aimed at improving equitable access to the Internet, providing seamless integrations with other technologies like browsers and OSes."
## Inspiration

We aimed to build smart cities by helping garbage truck drivers receive the most efficient route in near-real-time, minimizing waste buildup in prone areas. This approach directly addresses issues of inefficiency in traditional waste collection, which often relies on static daily or weekly routes.

You might be wondering, why not just let drivers follow standard routes? In densely populated areas, waste accumulates faster than in less populated zones. This means that a one-size-fits-all route approach leads to inefficient pickups, resulting in garbage buildup that contributes to air and water pollution. By targeting areas where waste accumulates more quickly, we can reduce contamination, improve air quality, and create greener, healthier environments. Additionally, optimizing waste collection can lead to more sustainable use of resources and reduce the strain on landfills.

## What it does

CleanSweep has sensors that collect real-time information about all the trash cans in the city, detecting different waste levels. This data is live-updated in the truck drivers' main portal while they drive, allowing them to receive the most efficient collection routes. These routes are optimized based on real-time data, including the percentage of trash bin capacity filled, traffic conditions between bins, and the number of days since the last pickup. As drivers collect trash, they can update their progress live to receive the next optimized route for the remaining bins.

## How we built it

For our hardware: We used two phone cameras in two similar scenarios that sent photos repeatedly to a computer. This image is then passed onto the Raspberry Pi where Python OpenCV was used to detect the trash level filled. Then a Node.JS script was used to display the levels (1 to 5) on a set of LEDs and then pass the information to a local server for further processing on the software side.

Backend: We received the reported trash level data from the hardware, information about the traffic in the surrounding vicinity as well as distance to the bin from the Google Maps API and Google Distance Matrix API, and pre-modeled values for time since last garbage retrieval and fed this into a Random Forest Classifier model on Databricks. Our model makes predictions on how to prioritize the bin routes in order to get a short path and distance–resulting in fewer emissions. An adjacency matrix was then used to retrieve the highest priority paths based on traffic and waste levels.

Frontend: We used React for the UI and TailwindCSS for styling to create the portal, deployed over Terraform. We brought all of this information together to display the recommended optimal routes based on how full the bins were (which can be changed by adding more trash to bin 1 and 2 in real life).

## Challenges we ran into

Measuring the level of the trash bin was actually a very challenging task for what seemed like a simple task. We first tried looking for pre-trained models that could do it for us, but other models only worked with object tracking. We went to OpenCV, but that only worked in certain conditions. Thus, one of the hardest challenges we experienced was making sure the lighting conditions, the setup for the cameras and hardware, and our OpenCV contour algorithm worked deterministically, as even a shadow could mess with our results.

## Accomplishments that we're proud of

None of us had ever worked with OpenCV or with using hardware at a hackathon. We wanted to take some of the skills we learned in our electrical classes and implement them with a cool solution, even if it was a bit tacky. We were super proud of learning how OpenCV worked and how we could use hardware as well as a network to create a complete interactive solution. Another major component of this hackathon was hardware as we’ve always wanted to use such a concept to visualize our project and make it easier to understand for everyone on what our goals are. We enjoyed our varied skills by stepping out of our comfort zone and using new technologies; each member used a new skill to implement in the project.

## What we learned

All of us learned about the parts of networks and hardware required to communicate including the security standards and communication frequency needed to keep all the data real time. We learned how to build a working website with a login system through the use of react and vite, as well as implementing the Google maps API for visualizing the route for the drivers. We also learned how GPIO works on a raspberry pi and how a simple JS script can control the power output for these pins to control LEDs. Another skill we found interesting was the use of OpenCV for image processing as it made computing the percentage a task of a second rather than a longer time of manual image processing algorithms. Finally, we were able to start using an ML model to train and help us find the best routes for drivers, this resulted in faster travel times as traffic was avoided and garbage collection optimized.

## What's next for Clean Sweep

Clean Sweep’s next goals would be to implement real time sensors in bins spread out all over the city to collect data and help train the model by providing the large amounts of data. This would then help implement this real time UI to efficiently visit all the necessary stops for the day. Such a system would also allow for coverage over rare cases such as a garbage truck missing a stop or allowing other trucks to take over a broken truck’s schedule as the route would update in real time to collect all garbage fast and efficiently.