Template variable pre-process concept
Preprocessing is just an optional step between defining a render array that will then use a twig template and the actual twig template.

It's a place where you can execute PHP code to prepare variables that you can later use in your template.

You don't have to use preprocess for you own templates but some things can only be done there or are a lot easier, as you can't execute arbitrary code in a twig template.

See template_preprocess_image() for an example. Note the different names of preprocess functions. There's always the one prefixed with template. That's reserved for the module that defines template in hook_theme() and is called first. And then it's called for each module and active theme, just like a hook, e.g. as yourmodule_preprocess_image.

That brings me to another advantage of the preprocess phase. There can only be one template that's actually used and while there are ways, modules in general shouldn't provide template unless they define it. But, every module can implement the preprocess function and influence the output of a template to a certain degree. For example to add a certain class or data attribute to a node so that some JS can then use it.