How would I wrap (or hard-code) HTML into a block title?
You can hard code in block template: "Drupal 7 Template Suggestions", so determine what module/delta/region appropriate for you.
Other way is using theme preprocessor.