Using python and machine learning to extract information from an invoice? Inital dataset? DISCLAIMER: I have absolutely no background with machine learning/data science, and am unfamiliar with the general lingo of data science, so please bear with me.

I'm trying to make a machine learning application with Python to extract invoice information (invoice number, vendor information, total amount, date, tax, etc.). As of right now, I'm using the Microsoft Vision API to extract the text from a given invoice image, and organizing the response into a top-down, line-by-line text document in hopes that it might increase the accuracy of my eventual machine learning model. My current situation is strictly using string parsing, and this method works pretty well for invoice number, data, and total amount. 

However, when it comes to vendor information (name, address, city, province, etc.) string parsing hardly ever works due to my script relying on the detection of an address (because addresses have a pretty distinct format compared to the rest of the information on an invoice). Tax information is difficult to parse because of the amount of numeric values that appear on an invoice.



So (and here's where I get lost)
I envision a machine learning model that will have an input of a single invoice image from a user, and it's output will be the extracted invoice information. I am currently looking into Azure Machine Learning Studio because the plug 'n play aspect of it appeals to me and it seems easy enough to use and experiment with. BUT I have no clue what the requirements are for an initial dataset! Should I just fill my dataset (in a CSV format btw) with the necessary information (invoice number, total amount, date, ...) from a bunch of sample invoices? If not, what other information should I include in my dataset? I was thinking x-y coordinate pairs of where the important information occurs on the image.

One last question related to this problem scope, which algorithm (regression, classification, clustering) could even "extract" (or help with it) information from the input text? As far as I know, regression predicts numeric values (i.e. 2 + x = 10 so 8 + x could be 16) and classification "labels" things (i.e. this image contains a tree, this one does not). I'm not too familiar with clustering, although I think it could be useful to identify the structure of the input text.

To summarize: what might be some features of an invoice that I can fill an initial dataset with to initialize a model? How could a clustering algorithm be used to identify the structure of an invoice? How could I utilize a regression/classification algorithm in this problem, and what inputs would these algorithms need to deliver meaningful results?

Any help/guidance/feedback would be appreciated to the moon and back. Sorry for my lack of knowledge, but this field is very interesting and need some help wrapping this all around my head. Thanks:)
I'm more of a beginner as well, but wanted to possibly help guide you towards next steps based on some of my experiences.

First, what you've illustrated here sounds a little along the lines of how a traditional EDI system does in Supply Chain &amp; Procurement processing, where it will receive a document and enter the information into transaction-based operational systems to create records as data. I'm not entirely sure how those work, and if they can only extract standardized formats from documents but that's something that you could possibly look into with a few searches.

Now as far as creating a CSV data set, that is a great idea for testing the accuracy of your algorithm on a set of invoices to train your model. Training a model is pretty self-explanatory, but essentially you'd be using a supervised machine learning strategy, where the system actively uses a training data set where the correct answers are known. By comparing the models results to the data set, you could then know if the algorithm is appropriately retrieving the information. I'd recommend that you include the information the application should be getting from each invoice, and compare that to what the machine did retrieve. That way you effectively known the error of the model. (x,y) coordinates would work well for preliminary attempts to get a better feel for location-based assessment of the invoices, so that's a pretty good idea in my opinion as well.

Regarding algorithm types, this one will almost always be "it depends". Many times, I'll use a couple and compare the error that returns for each model to find the one that works the best for what I'm predicting. That said, Clustering can be useful for continuous, numerical data and "grouping" items based on the location of the coordinates. I think clustering could be useful if you're breaking down the coordinates of the information on an image, but wouldn't be of much use to gather and extract the text from the invoice. And to not go too deep, K-Means would probably not be great at all because the locations for the cluster centroids is entirely randomized and you must also declare a number of n clusters, which would be impracticable for the purpose at hand. 

I'd recommend looking into the different types of Neural Networks(NN's). These are used extensively to take images of some kind, and break them down to best estimate the contents of an unknown input. It's the type of algorithm Google uses to do image searches and things along those lines. It's also the type of software that tells an autonomous vehicle that the sign ahead is a stop sign. There are different types of NN's out there, and it can get pretty overwhelming pretty quickly, so I recommend looking into basic videos to explain it. If you do decide to go this route, Google Cloud has a pretty solid platform with there google.vision and other machine-learning-in-a-box libraries that are really easy to import and play with.

as far as inputs, define the information that you want extracted, and record that data for use of testing and training as I mentioned above. Then play around with a few methods in sciket-learn and TensorFlow!

Unfortunately, I can't give you a straight-forward answer on the many trade-offs with different algorithms or inputs, but I hope that this gave you a few ideas about next steps or provided you with a better understanding of how you can begin the project. Good luck!