Custom block template
An array with #markup is a render array, just the most basic version of it. If you want to use a theme or template, then just do so. Using and defining templates hasn't changed that much compared to D7, the biggest difference is that they're now html.twig files and not .tpl.php.