How do you change the location of `form_set_error` output?
Extending the (correct) answer from Clive, I worked through the IFE code. I didn't really need an entire module dedicated to this, so I adopted a few snippets here and there to get the result I needed. I've marked his answer as correct because it is ultimately the right answer. 

The code is below, all credit goes to Clive and the IFE team - I just wanted to present the simplified version for anyone looking for a similar answer. 

// Standard gear - do some custom validation and set the errors
// as you go..
// 
// Once all the validation has been done, call MODULE_errors_reset
// which will return an array of all errors against their ID. 
// Expose this array however you like to your template, or loop
// over your form adding a #suffix to each element with an error
//
function MODULE_form_name_validate($form, &amp;$form_state) {
    drupal_set_message("validating...");

    form_set_error("description", "There is an error here!!!!");
    form_set_error("tags", "Yep, and here too!!!");

    $reset_errors = MODULE_errors_reset( $form );

    drupal_set_message( "&lt;pre&gt;" . print_r( $reset_errors, true ) . "&lt;/pre&gt;" );
}

// This part is adopted from IFE. It's changed in two ways, it returns
// an array (which also merges with its recursive self). 
// And it also skips all the 'display' stuff present in the original
// Essentially it extracts out the error messages from the session and unsets 
// them. I am assuming that Drupal 7 marks the success of a validation based not
// whether the SESSION variable contains anything, the SESSION seems to be only
// for the message at the top of the screen.
//
function MODULE_errors_reset( $element ) {
    if( ! isset( $_SESSION[ 'messages' ] ) ) {
        return;
    }

    $reset_errors = array();

    // Recurse through all children.
    foreach( element_children( $element ) as $key ) {
        if( isset( $element[ $key ] ) &amp;&amp; $element[ $key ] ) {
            $reset_errors += MODULE_errors_reset( $element[ $key ] );
        }
    }

    // Check for errors and settings
    $errors = form_get_errors();
    $element_id = implode( '][', $element[ '#parents' ] );

    if ( !empty( $errors[ $element_id ] )) {
        $error_message = $errors[ $element_id ];

        // Get error id
        $error_id = array_search( $error_message, $_SESSION[ 'messages' ][ 'error' ] );

        if( $error_id !== FALSE ) {
            unset( $_SESSION[ 'messages' ][ 'error' ][ $error_id ] );
            $_SESSION[ 'messages' ][ 'error' ] = array_values( $_SESSION[ 'messages' ][ 'error' ]  );

            if( count( $_SESSION[ 'messages' ][ 'error' ] ) &lt;= 0 ) {
                unset( $_SESSION[ 'messages' ][ 'error' ] );
            }

            $reset_errors[ $element[ '#id' ] ] = $error_message;
        }
    }

    return $reset_errors;
}

// If there are no form errors, we still hit here, even after the 'reset', this is
// a good thing. 
function MODULE_form_name_submit( $form, &amp;$form_submit ) {
    drupal_set_message("submited!");
}