How do I use Machine Learning to analyze process statistics of automated test results?
I want to come up with a script/tool(Python) that leverages machine learning and automatically analyze the results for me, instead of me looking at the results manually every day.


This is the wrong approach. Choose your tools to solve tasks, not because they sound cool. Imagine a craftsman saying "I want to use a hammer". (However, machine learning might still suit to this task.)


  From what I read online, I think I need to [...] train it for a few days.


Depends on how much data, how much features, how much computing power and which algorithm you use. In many cases you can train a model within seconds on a normal computer.

The real question

The problem: You have some characteristics (features) like CPU usage, memory usage, ... for every automated test. For some instances (meaning a test and the associated features) you have the binary information "normal", "abnormal". Now, when you get the characteristics of a new test, you want to automatically decide if it was normal or abnormal.

This is a binary classification problem. If you have the label ("normal", "abnormal") for all instances, you can apply supervised learning methods. If you have the label only for some instances, it is semi-supervised learning. If you don't have any labels it is unsupervised.

The Answer

I assume you are in a supervised learning case. Then, I would suggest to use a decision tree (see sklearn.tree.DecisionTreeClassifier - sklearn is pretty easy to use and has many standard algorithms implemented) for this decision making. It basically automatically generates many nested if/else statements to get to the decision. Hence it is interpretable and you could manually adjust or improve it.