## Inspiration

raj

## What it does

it takes an input image of handwritten numbers and uses an SVM to detect the numbers

## How I built it

used opencv and scikit-learn python libraries

## Challenges I ran into

computer vision

## Accomplishments that I'm proud of

detecting handwriting from own images

## What I learned

Machine Learning and computer vision

## What's next for Row Reducer

android app
## Inspiration

Through some initial tests, we saw that the Google Vision API was decent at optical character recognition (OCR), and we decided we would leverage that to make data entry as simple as taking a picture.

## What it does

Detects tabular structure in images of handwritten/typed text, and converts it either to CSV or a Google Sheets spreadsheet.

## How we built it

We built it using React-Express-Node stack, hosted on the Google Cloud App Engine. The app uses a Google Cloud Firestore to save scanned spreadsheets, the Google AI Vision API for OCR and Google OAuth for Google logins.

## Challenges we ran into

The unpredictable order in which the Vision API recognizes text in an image made it a challenge to write a flexible algorithm capable of conserving the positional order of the (assumed) structured data. Also turns out that React does not play nice when trying to convert and uploaded image into a base64-encoded Buffer consumable by the Google Vision API.

## Accomplishments that we're proud of

Gettting the algorithm to work properly.

## What we learned

That ML + JavaScript is a perfectly good combination, and the Google Cloud stack is pretty good for developing scalable apps at lightning speed.

## What's next for SVS

Leveraging the Google Document Understanding API to expand the use case of the app, such as creating entire documents with freeform data straight from pictures.
## What Inspired us

Due to COVID, many students like us have become accustomed to work on their schoolwork, projects and even hackathons remotely. This led students to use online resources at their disposal in order to facilitate their workload at home. One of the tools most used is “ctrl+f” which enables the user to quickly locate any text within a document. Thus, we came to a realisation that no such accurate method exists for images. This led to the birth of our project for this hackathon titled “PictoDocReader”.

## What you learned

We learned how to implement Dash in order to create a seamless user-interface for Python. We further learnt several 2D and 3D pattern matching algorithms such as, Knuth-Morris-Pratt, Bird Baker, Karp and Rabin and Aho-Corasick. However, only implemented the ones that led to the fastest and most accurate execution of the code.
Furthermore, we learnt how to convert PDFs to images (.png). This led to us learning about the colour profiles of images and how to manipulate the RGB values of any image using the numpy library along with matplotlib. We also learnt how to implement Threading in Python in order to run tasks simultaneously. We also learnt how to use Google Cloud services in order to use Google Cloud Storage to enable users to store their images and documents on the cloud.

## How you built your project

The only dependencies we required to create the project were PIL, matplotlib, numpy, dash and Google Cloud.

**PIL** - Used for converting a PDF file to a list of .png files and manipulating the colour profiles of an image.

**matplotlib** - To plot and convert an image to its corresponding matrix of RGB values.

**numpy** - Used for data manipulation on RGB matrices.

**dash** - Used to create an easy to use and seamless user-interface

**Google Cloud** - Used to enable users to store their images and documents on the cloud.

All the algorithms and techniques to parse and validate pixels were all programmed by the team members. Allowing us to cover any scenario due to complete independence from any libraries.

## Challenges we faced

The first challenge we faced was the inconsistency between the different RGB matrices for different documents. While some matrices contained RGB values, others were of the form RGBA. Therefore, this led to inconsistent results when we were traversing the matrices. The problem was solved using the slicing function from the numpy library in order to make every matrix uniform in size.

Trying to research best time complexity for 2d and 3d pattern matching algorithms. Most algorithms were designed for square images and square shaped documents. While we were working with any sized images and documents. Thus, we had to experiment and alter the algorithms to ensure they worked best for our application.

When we worked with large PDF files, the program tried to locate the image in each page one by one. Thus, we needed to shorten the time for PDFs to be fully scanned to make sure our application performs its tasks in a viable time period. Hence, we introduced threading into the project to reduce the scanning time when working with large PDF files as each page was scanned simultaneously. Although we have come to the realisation that threading is not ideal as the multi-processing greatly depends on the number of CPU cores of the user’s system. In an ideal world we would implement parallel processing instead of threading.