Programmatically loading Custom block in page/node preprocessor
I think the problem here is that Drupal 8 has both block definitions (a.k.a. plugins) and block instances.  Both of which tend to be called blocks.  What is listed in the custom block library is a block definition. This is like what is done when a block is created with code.  It define the block but not the configuration.

An instance is the configuration data for a block.  It has the settings for a specific block.  E.g., title override, display title, region, and the like.

So, you are trying to render the block definition with code that requires an instance block.

How do you get an instance of a block to render?  The simplest way is to create a hidden region in your theme by adding a region line in the info file.  Something like "Hidden Blocks".

Add your custom (or code based) blocks to this region in the block placement screen.  I also tend to set the rules for where this can be displayed just to help keep these from being generated when not needed.

Once you do this, you should be able to render using the block id from the placement screen.  Note this can be different from the plug-in id on the custom block page if you have placed it more than one.