Automated Function (python)
Your code looks decent, although this is not really the place to ask these types of questions. You're looking to write a wrapper around SK-Learn,

What I would try to do is to write a method to handle different types of datasets(json or csv) and then call it on a filepath and then have a function which reads in a set of models and applies each of them to the dataset in question, and finally uses metrics present in sklearn to get some results.

There is auto-sklearn which does this for you and they have a pretty solid codebase that is well documented. Here is a link for your convenience (https://automl.github.io/auto-sklearn/stable/)