How to include SVG in a block markup?
Use inline_template instead of markup, or a theme function (as suggested by Ivan Jaros in a comment also).
Refer to class InlineTemplate for way more details. Here is a partial quote of it:

Provides a render element where the user supplies an in-line Twig template.
Properties:

#template: The inline Twig template used to render the element.
#context: (array) The variables to substitute into the Twig template. Each variable may be a string or a render array.

Usage example:
$build['hello']  = [
  '#type' =&gt; 'inline_template',
  '#template' =&gt; &quot;{% trans %} Hello {% endtrans %} &lt;strong&gt;{{name}}&lt;/strong&gt;&quot;,
  '#context' =&gt; [
    'name' =&gt; $name,
  ]
];