## Inspiration

We were going to build a themed application to time portal you back to various points in the internet's history that we loved, but we found out prototyping with retro looking components is tough. Building each component takes a long time, and even longer to code. We started by automating parts of this process, kept going, and ended up focusing all our efforts on automating component construction from simple Figma prototypes.

## What it does

Give the plugin a Figma frame that has a component roughly sketched out in it. Our code will parse the frame and output JSX that matches the input frame. We use semantic detection with Cohere classify on the button labels combined with deterministic algorithms on the width, height, etc. to determine whether a box is a button, input field, etc. It's like magic! Try it!

## How we built it

Under the hood, the plugin is a transpiler for high level Figma designs.

Similar to a C compiler compiling C code to binary, our plugin uses an abstract syntax tree like approach to parse Figma designs into html code.

Figma stores all it's components (buttons, text, frames, input fields, etc) in nodes.. Nodes store properties about the component or type of element, such as height, width, absolute positions, fills, and also it's children nodes, other components that live within the parent component. Consequently, these nodes form a tree.

Our algorithm starts at the root node (root of the tree), and traverses downwards. Pushing-up the generated html from the leaf nodes to the root.

The base case is if the component was 'basic', one that can be represented with two or less html tags. These are our leaf nodes. Examples include buttons, body texts, headings, and input fields. To recognize whether a node was a basic component, we leveraged the power of LLM.

We parsed the information stored in node given to us by Figma into English sentences, then used it to train/fine tune our classification model provided by co:here. We decided to use an ML to do this since it is more flexible to unique and new designs. For example, we were easily able to create 8 different designs of a destructive button, and it would be time-consuming relative to the length of this hackathon to come up with a deterministic algorithm. We also opted to parse the information into English sentences instead of just feeding the model raw figma node information since the LLM would have a hard time understanding data that didn't resemble a human language.

At each node level in the tree, we grouped the children nodes based on a visual hierarchy. Humans do this all the time, if things are closer together, they're probably related, and we naturally group them. We achieved a similar effect by calculating the spacing between each component, then greedily grouped them based on spacing size. Components with spacings that were within a tolerance percentage of each other were grouped under one html

 . We also determined the alignments (cross-axis, main-axis), of these grouped children to handle designs with different combinations of orientations. Finally, the function is recursed on their children, and their converted code is pushed back up to the parent to be composited, until the root contains the code for the design. Our recursive algorithm made it so our plugin was flexible to the countless designs possible in Figma.
## Challenges we ran into

We ran into three main challenges. One was calculating the spacing. Since while it was easy to just apply an algorithm to merge two components at a time (similar to mergesort), it would produce too many nested divs, and wouldn't really be useful for developers to use the created component. So we came up with our greedy algorithm. However, due to our perhaps mistaken focus on efficiency, we decided to implement a more difficult O(n) algorithm to determine spacing, where n is the number of children. This sapped a lot of time away, which could have been used for other tasks and supporting more elements.

The second main challenge was with ML. We were actually using Cohere Classify wrongly, not taking semantics into account and trying to feed it raw numerical data. We eventually settled on using ML for what it was good at - semantic analysis of the label, while using deterministic algorithms to take other factors into account. Huge thanks to the Cohere team for helping us during the hackathon! Especially Sylvie - you were super helpful!

We also ran into issues with theming on our demo website. To show how extensible and flexible theming could be on our components, we offered three themes - windows XP, 7, and a modern web layout. We were originally only planning to write out the code for windows XP, but extending the component systems to take themes into account was a refactor that took quite a while, and detracted from our plugin algorithm refinement.

## Accomplishments that we're proud of

We honestly didn't think this would work as well as it does. We've never built a compiler before, and from learning off blog posts about parsing abstract syntax trees to implementing and debugging highly asychronous tree algorithms, I'm proud of us for learning so much and building something that is genuinely useful for us on a daily basis.

## What we learned

Leetcode tree problems actually are useful, huh.

## What's next for wayback

More elements! We can only currently detect buttons, text form inputs, text elments, and pictures. We want to support forms too, and automatically insert the controlling componengs (eg. useState) where necessary.
## Inspiration

We, as passionate tinkerers, understand the struggles that come with making a project come to life (especially for begineers). **80% of U.S. workers agree that learning new skills is important, but only 56% are actually learning something new**. From not knowing how electrical components should be wired, to not knowing what a particular component does, and what is the correct procedure to effectively assemble a creation, TinkerFlow is here to help you ease this process, all in one interface.

## What it does

-> Image identification/classification or text input of available electronic components
-> Powered by Cohere and Groq LLM, generates wiring scheme and detailed instructions (with personality!) to complete an interesting project that is possible with electronics available
-> Using React Flow, we developed our own library (as other existing softwares were depreciated) that generates electrical schematics to make the fine, precise and potentially tedious work of wiring projects easier.
-> Display generated text of instructions to complete project

## How we built it

We allowed the user to upload a photo, have it get sent to the backend (handled by Flask), used Python and Google Vision AI to do image classification and identify with 80% accuracy the component.

To provide our users with a high quality and creative response, we used a central LLM to find projects that could be created based on inputted components, and from there generate instructions, schematics, and codes for the user to use to create their project. For this central LLM, we offer two options: Cohere and Groq. Our default model is the Cohere LLM, which using its integrated RAG and preamble capability offers superior accuracy and a custom personality for our responses, providing more fun and engagement for the user. Our second option Groq though providing a lesser quality of a response, provides fast process times, a short coming of Cohere. Both of these LLM's are based on large meticulously defined prompts (characterizing from the output structure to the method of listing wires), which produce the results that are necessary in generating the final results seen by the user.

In order to provide the user with different forms of information, we decide to present electrical schematics on the webpage. However during the development due to many circumstances, our group had to use simple JavaScript libraries to create its functionality.

## Challenges we ran into

* LLM misbehaving: The biggest challenge in the incorporation of the Cohere LLM was the ability to generate consistent results through the prompts used to generate the results needed for all of the information provided about the project proposed. The solution to this was to include a very specifically defined prompts with examples to reduce the amount of errors generated by the LLM.
* Not able to find a predefined electrical schematics library to use to generate electrical schematics diagrams, there we had start from scratch and create our own schematic drawer based on basic js library.

## Accomplishments that we're proud of

Create electrical schematics using basic js library.
Create consistent outputting LLM's for multiple fields.

## What we learned

Ability to overcome troubles - consistently innovating for solutions, even if there may not have been an easy route (ex. existing library) to use - our schematic diagrams were custom made!

## What's next for TinkerFlow

Aiming for faster LLM processing speed.
Update the user interface of the website, especially for the electrical schematic graph generation.
Implement the export of code files, to allow for even more information being provided to the user for their project.
## Inspiration

Our major source of motivation for leafy was an incident that took place recently. My mother and her friends are very enthusiastic about botany. My house has a big garden with innumerable plants. She usually spends a lot of her time catering to all their needs. Once my neighbour’s plant recently caught a infectious plant disease which was left undiagnosed. Due to this, the disease got communicated to our plants leading to their gradual death. Being nature lovers, it was a very traumatic experience for all of us. It made me wonder about the huge losses that the agriculture sector incurs because of the lack of right awareness at the right time. This prompted me to prepare a mobile application to prevent its reoccurrence.

## What it does

The backend uses a well trained CNN based model which is trained by dataset comprising of over 30000 plant images. It can accurately predict and classify leaf diseases to a degree of 94%. If the model finds the detected disease to be infectious, it notifies all the nearby mobile phones(in radius < 2 km) to take precautions to prevent the disease.

## How we built it

CNN model using Tensorflow, Scikit-learn, and OpenCV
App using android studio
API using flask(python)
and around 5 cups of coffee ☕️

## Challenges we ran into

We used our knowledge of java, python and html with a bit of css. Building a CNN model which could private the highest level of accuracy and F1 score was the main challenge for us. Another challenge was trying and testing the perfect combination of the CNN model parameters achieved desired results.

## Accomplishments that we're proud of

Through the course of this hackathon, we learnt the use of Google cloud AI notebooks which will go a long way in helping us become better hackers. The fact that we were able to document our imagination and ideas into code successfully and complete this project in time is probably a big accomplishment for both of us.

## What's next for Leafy

Just like we build Leafy for leaves, we can extend and develop another model which would be able to analyse soil through photos and predict about any lack of nutrients.

Room No:172
Team Name: MTC Hackers