Removing input formatting options and description from comment form
Ok found this drupal issue discussing the matter: Node and Comment Input Format fieldset is not working (I found it by searching for "edit-comment-body-und-0-format" which html id of the fieldset that contains all the input format mumbo jumbo).

At the end of this issue is a link to the following site which gives a solution:
How To Remove Format Options / Guideliness From Comments Textarea in Drupal 7.

Here's the code I'm using on my site, in template.php of my theme (I'm using a zen subtheme):

function mytheme_form_comment_form_alter(&amp;$form, &amp;$form_state) {

  $form['comment_body']['#after_build'][] = 'mytheme_customize_comment_form';

}

function mytheme_customize_comment_form(&amp;$form) {
  $form[LANGUAGE_NONE][0]['format']['#access'] = FALSE;
  return $form;
}


I just tried it and it does seem to be working. Woohoo!