## Inspiration

The day to day process of navigating to websites and then adding products to the cart is time-consuming. So We thought of making a voice-enabled shopping assistant which can search and add products to the cart just with a voice command.

## What it does

It can search and add products to the cart just with a voice command. It is also capable of performing a day-to-day activity such as playing music, opening applications, etc

## How I built it

We have built it in python using the selenium library and performing web automation

## Challenges I ran into

Web automation was really tough

## Accomplishments that I'm proud of

The final product which performs on voice command is an accomplishment.

## What I learned

I have learned many new skills like web automation, python libraries, etc

## What's next for Voice Shop - Voice-enabled Shopping Assistant
## Inspiration

Let’s face it: getting your groceries is hard. As students, we’re constantly looking for food that is healthy, convenient, and cheap. With so many choices for where to shop and what to get, it’s hard to find the best way to get your groceries. Our product makes it easy. It helps you plan your grocery trip, helping you save time and money.

## What it does

Our product takes your list of grocery items and searches an automatically generated database of deals and prices at numerous stores. We collect this data by collecting prices from both grocery store websites directly as well as couponing websites.

We show you the best way to purchase items from stores nearby your postal code, choosing the best deals per item, and algorithmically determining a fast way to make your grocery run to these stores. We help you shorten your grocery trip by allowing you to filter which stores you want to visit, and suggesting ways to balance trip time with savings. This helps you reach a balance that is fast and affordable.

For your convenience, we offer an alternative option where you could get your grocery orders delivered from several different stores by ordering online.

Finally, as a bonus, we offer AI generated suggestions for recipes you can cook, because you might not know exactly what you want right away. Also, as students, it is incredibly helpful to have a thorough recipe ready to go right away.

## How we built it

On the frontend, we used **JavaScript** with **React, Vite, and TailwindCSS**. On the backend, we made a server using **Python and FastAPI**.

In order to collect grocery information quickly and accurately, we used **Cloudscraper** (Python) and **Puppeteer** (Node.js). We processed data using handcrafted text searching. To find the items that most relate to what the user desires, we experimented with **Cohere's semantic search**, but found that an implementation of the **Levenshtein distance string algorithm** works best for this case, largely since the user only provides one to two-word grocery item entries.

To determine the best travel paths, we combined the **Google Maps API** with our own path-finding code. We determine the path using a **greedy algorithm**. This algorithm, though heuristic in nature, still gives us a reasonably accurate result without exhausting resources and time on simulating many different possibilities.

To process user payments, we used the **Paybilt API** to accept Interac E-transfers. Sometimes, it is more convenient for us to just have the items delivered than to go out and buy it ourselves.

To provide automatically generated recipes, we used **OpenAI’s GPT API**.

## Challenges we ran into

Everything.

Firstly, as Waterloo students, we are facing midterms next week. Throughout this weekend, it has been essential to balance working on our project with our mental health, rest, and last-minute study.

Collaborating in a team of four was a challenge. We had to decide on a project idea, scope, and expectations, and get working on it immediately. Maximizing our productivity was difficult when some tasks depended on others. We also faced a number of challenges with merging our Git commits; we tended to overwrite one anothers’ code, and bugs resulted. We all had to learn new technologies, techniques, and ideas to make it all happen.

Of course, we also faced a fair number of technical roadblocks working with code and APIs. However, with reading documentation, speaking with sponsors/mentors, and admittedly a few workarounds, we solved them.

## Accomplishments that we’re proud of

We felt that we put forth a very effective prototype given the time and resource constraints. This is an app that we ourselves can start using right away for our own purposes.

## What we learned

Perhaps most of our learning came from the process of going from an idea to a fully working prototype. We learned to work efficiently even when we didn’t know what we were doing, or when we were tired at 2 am. We had to develop a team dynamic in less than two days, understanding how best to communicate and work together quickly, resolving our literal and metaphorical merge conflicts. We persisted towards our goal, and we were successful.

Additionally, we were able to learn about technologies in software development. We incorporated location and map data, web scraping, payments, and large language models into our product.

## What’s next for our project

We’re very proud that, although still rough, our product is functional. We don’t have any specific plans, but we’re considering further work on it. Obviously, we will use it to save time in our own daily lives.
## Inspiration

We've worked on e-commerce stores (Shopify/etc), and managing customer support calls was tedious and expensive (small businesses online typically have no number for contact), despite that ~60% of customers prefer calls for support questions and personalization. We wanted to automate the workflow to drive more sales and save working hours.

Existing solutions require custom setup in workflows for chatbots or asking; people still have to answer 20 percent of questions, and a lot are confirmation questions (IBM). People have question fatigue with bots to get to an actual human.

## What it does

It's an embeddable javascript widget/number for any e-commerce store or online product catalog that lets customers call, text, or message on site chat about products personalized to them, processing returns, and general support. We plan to expand out of e-commerce after signing on 100 true users who love us. Boost sales while you are asleep instead of directing customers to a support ticket line.

We plan to pursue routes of revenue with:

* % of revenue from boosted products
* Monthly subscription
* Costs savings from reduced call center capacity requirements

## How we built it

We used a HTML/CSS frontend connected to a backend of Twilio (phone call, transcription, and text-to-speech) and OpenAI APIs (LLMs, Vector DBQA customization).

## Challenges we ran into

* Deprecated Python functionality for Twilio that we did not initially realize, eventually discovered this while browsing documentation and switched to JS
* Accidentally dumped our TreeHacks shirt into a pot of curry

## Accomplishments that we're proud of

* Developed real-time transcription connected to a phone call, which we then streamed to a custom-trained model -- while maintaining conversational-level latency
* Somehow figured out a way to sleep
* Became addicted to Pocari Sweat

## What we learned

We realized the difficulty of navigating documentation while traversing several different APIs. For example, real-time transcription was a huge challenge.

Moreover, we learned about embedding functions that allowed us to customize the LLM for our use case. This enabled us to provide a performance improvement to the existing model while also not adding much compute cost. During our time at TreeHacks, we became close with the Modal team as they were incredibly supportive of our efforts. We also greatly enjoyed leveraging OpenAI to provide this critical website support.

## What's next for Ellum

We are releasing the service to close friends who have experienced these problems, particularly e-commerce distributors and beta-test the service with them. We know some Shopify owners who would be down to demo the service, and we hope to work closely with them to grow their businesses.

We would love to pursue our pain points even more for instantly providing support and setting it up. Valuable features, such as real-time chat, that can help connect us to more customers can be added in the future. We would also love to test out the service with brick-and-mortar stores, like Home Depot, Lowes, CVS, which also have a high need for customer support.

Slides: <https://drive.google.com/file/d/1fLFWAgsi1PXRVi5upMt-ZFivomOBo37k/view?usp=sharing>

Video Part 1: <https://youtu.be/QH33acDpBj8>
Video Part 2: <https://youtu.be/gOafS4ZoDRQ>