How do I dynamically modify a block's mark up at render time, in Drupal 7?
I think you are looking for hook_block_view_alter(). It provides both the alterable $data, as well as $block, which contains the module and delta.

To determine which blocks are available, you could start off with:

function MYTHEME_block_view_alter(&amp;$data, $block) {
  kpr($block-&gt;module);
  kpr($block-&gt;delta);
}


Then once you find the module:

/**
 * Implements hook_block_view_alter().
 */
function YOURTHEME_block_view_alter(&amp;$data, $block) {
  if ($block-&gt;module == 'block' &amp;&amp; $block-&gt;delta == 1) {
    $data['content'] = '&lt;p&gt;New content&lt;/p&gt;';
  }
}


Or even:

/**
 * Implements hook_block_view_MODULE_DELTA_alter().
 */
function YOURTHEME_block_view_block_4_alter(&amp;$data, $block) {
  $data['content'] = '&lt;p&gt;New content&lt;/p&gt;';
}