How do I unset a block? I just started creating a Drupal 8 theme. 

For example in Drupal 7 if I want to remove the breadcrumb, I just need to remove this line of code &lt;?php if ($breadcrumb) print $breadcrumb; ?&gt; from the page.tpl.php. 

In Drupal 8 I removed {{ page.breadcrumb }} from the page.html.twig and I still seeing the breadcrumb. So not sure how to remove or unset these kind of blocks in Drupal 8.
Yes, what you are seeing is the render cache. If you are developing a theme, you likely want to disable render caching and enable twig debug. See sites/example.settings.local.php for instructions on how to do that.

That said, breadcrumbs, like almost everything else, is a block now. Blocks are configuration. If you don't want a block, just remove that block in the UI. If you just remove it in the template, Drupal still needs to load and prepare that block for rendering as it cannot know the template doesn't print it.

And if you don't display the region (you always display regions with 0-N blocks, not directly blocks in page template), you should also not define it as an available region so that the user/site admin knows that he can't place blocks there.