PHP Class that handles all includes / required files
Mixing

You are mixing 2 things:


class loading
"template" loading


You aren't over complicating things - you try to over simplify them.

Class loading

In your solution you are loading all classes if only (for example) 2 of them required to serve the request the loading of the other 52321 class is just a waste of time and resources.

Read about automatic class loading: Autoloading Classes

With this feature in your mind you can create a custom logic to load the necessary classes (name resolution, class location in file system etc.).

Template

The other thing what you want is loading your templates into the rendering process. This is hard to do it right, all depends on your "framework" solutions for example naming conventions, is your app frame an MVC based stuff etc.