Broken custom font mesh
Fixed :)
For every one else with this problem:
Previous attempts to use different format was a good idea, but WOFF was not supported, so I found out that blender supports also OTF. I downloaded pack from Fontsquirrel (otf by default) instead Gooogle fonts and it works :) Converting TTF to OTF would probably work too, but I did not tried this.
So short answer - use OTF instead TTF because TTF in Blender is buggy.

Edit: I found out that font format is not the case. Blender cannot handle glyphs in fonts with overlapping shapes (so for ex. &quot;X&quot; should be single mesh and not two crossed boxes one on top of another one). In my case same font from other source was fixed.
But we can fix this manually (merge shapes of glyph) in any font editor like glyphrstudio.com (online) or FontForge (desktop).