Create a reference all available block content
Like I said, I'm new to Drupal 8 so not familiar with new rule and features. After a bit digging here is what I found out. 

Entity Reference field allows to reference block instances (config) or custom blocks (content). 

Before I was set my field to Content > Custom block, now I switch it to Configuration > Block then you have to Place the block that you want to reference to, even if it is in disabled. After that, you can have your Entity Reference field pick that up otherwise it won't show.

You can have a look at here for more detail.