Can't drupal_execute('filter_admin_format_form') on hook_install?
You can execute that code, but you need to give a unique name for the input format being saved. 

As alternative, if you want to be sure there is an input format called Full HTML, you can save the input format in the database using db_query(), and using the code of filter_admin_format_form_submit(), the form submission hander for the form you are executing, as guideline.
The involved database tables are two: filter_formats, and filter_formats; the first contains the data about the input format, and the latter contains the data about the input filters associated with an input format. The relevant executed queries are the following ones.

  if (!$format) {
    $new = TRUE;
    db_query("INSERT INTO {filter_formats} (name) VALUES ('%s')", $name);
    $format = db_result(db_query("SELECT MAX(format) AS format FROM {filter_formats}"));
    drupal_set_message(t('Added input format %format.', array('%format' =&gt; $name)));
  }




db_query("DELETE FROM {filters} WHERE format = %d", $format);




  foreach ($form_state['values']['filters'] as $id =&gt; $checked) {
    if ($checked) {
      list($module, $delta) = explode('/', $id);
      // Add new filters to the bottom.
      $weight = isset($current[$id]-&gt;weight) ? $current[$id]-&gt;weight : 10;
      db_query("INSERT INTO {filters} (format, module, delta, weight) VALUES (%d, '%s', %d, %d)", $format, $module, $delta, $weight);

      // Check if there are any 'no cache' filters.
      $cache &amp;= !module_invoke($module, 'filter', 'no cache', $delta);
    }
  }


In Drupal 7, it would be easier, as you can call filter_format_save(), which always save the input format passed as argument, and eventually merge the format you passed as argument with the existing one.  

The function that shows the error you are noticing is the form validation handler,  filter_admin_format_form_validate(), which contains the following code.

if ($result) {
  form_set_error('name', t('Filter format names need to be unique. A format named %name already exists.', array('%name' =&gt; $name)));
}


As the form validation handlers, and the form submission handlers are executed when you call drupal_execute(), the same error is returned, even when drupal_execute('filter_admin_format_form', $form_state, $arg) is called from hook_init(); it just happens that the error message shown with form_set_error() is not immediately visible when hook_init() is invoked, but the input format is not saved. 
To notice that filter_admin_format_form_submit() executes the following code.

  $format = isset($form_state['values']['format']) ? $form_state['values']['format'] : NULL;
  $current = filter_list_format($format);
  $name = trim($form_state['values']['name']);
  $cache = TRUE;

  // Add a new filter format.
  if (!$format) {
    $new = TRUE;
    db_query("INSERT INTO {filter_formats} (name) VALUES ('%s')", $name);
    $format = db_result(db_query("SELECT MAX(format) AS format FROM {filter_formats}"));
    drupal_set_message(t('Added input format %format.', array('%format' =&gt; $name)));
  }
  else {
    drupal_set_message(t('The input format settings have been updated.'));
  }


Supposing that the form validation handler doesn't return an error, the form submission handler is not saving any data in the "filter_formats" table, if the input format is not new; in such case, the function executes the else part of the control statement, which just output a message.