How to add field to fieldset by ajax?
First - ajax callback should not make any persistent changes to $form. It's only duty is to return proper part of form array. Render all elements in form creation function:

mymodule_form($form, &amp;$form_state) {
  $form['image'] = array (
    '#type' =&gt; 'fieldset',
    '#title' =&gt; t('images'),
    '#collapsible' =&gt; TRUE,
    '#collapsed' =&gt; FALSE,
    '#prefix' =&gt; '&lt;div id="product_images"&gt;',
    '#suffix' =&gt; '&lt;/div&gt;',
  );
  if (empty($form_state['image']['count'])) {
    // Keep number of fields either in form or form state, both works.
    $form_state['image']['count'] = 4; // Default number of elements
  }
  elseif( isset($form_state['triggering_element']) 
      and $form_state['triggering_element']['#machine_name'] == 'more_images_button' ) {
    // If form was triggered by proper form element, increase counter
    $form_state['image']['count']++;
  }
  // Loop to create elements
  for ($i = 0; $i &lt; $form_state['image']['count']; $i++) 
  {
    $form['image'][$i] = array (
      '#markup' =&gt; "&lt;p&gt;$i&lt;/p&gt;",
    );
  }
  // And our button
  $form['image']['more'] = array (
    '#type' =&gt; 'button',
    '#value' =&gt; t('More'),
    '#ajax' =&gt; array (
      'callback' =&gt; 'product_more_images',
      'wrapper' =&gt; 'product_images',
      'method' =&gt; 'replace',
      'effect' =&gt; 'fade',
      'prevent' =&gt; 'submit',
    ),
    '#machine_name' =&gt; 'more_images_button',
  );
}

// Now callback function is trivial
function product_more_images($form, $form_state) {
  return $form['image'];
}


Of course it's only a sketch, with all off topic parts removed.