Remove Foundation Concrete Block
Yep, you need to worry about the structure. You may be fine with a pressure treated box-frame inside the void, if the section isn't directly load-bearing. 

But typically, a proper steel angle lintel is required and that must run at least over half of the neighbor block width. A call to a Mason might surprise you with how cheap it'll be for them to knock it out.