Beside drive train, what is the difference between group components (e.g. 105 vs. Ultegra)?
Since you don't mention specific group sets (ie 6700, 6600, etc.) I'll say the primary differences are going to be weight and feel and to a lesser extent performance.

Ultegra 6700 levers are carbon fiber with a 'better' shape that some say offers better brake feel. Also the pivot point is different on 6700 vs 5700 (105) as it has trickled down from the Dura Ace design which theoretically gives more leverage and also requires less finger pull at the beginning of the stroke to move the brake pads when compared to 105 that don't have this feature yet.

As for the brakes themselves I believe the only noticeable difference is weight.