Close Fieldset when another is opened
Sounds like a case for a jquery.ui accordion fieldgroup setup instead.

e.g: If you install https://drupal.org/project/field_group

I believe you can also use those group styles in custom FAPI forms. At least, I've done that for vertical tab, but maybe that's core now...

You could also add the javascript manually with

drupal_add_library('system', 'ui.accordion');


and use it as per: http://jqueryui.com/accordion/