How do you evaluate a wheelset?
I'm looking for an improvement in performance in aerodynamics, weight and reduced rolling resistance.


If this is what you want then you can pay for light hubs with high quality bearings that give you low rolling resistance.  The rest of rolling resistance is due to your choice of tire and tire pressure.  Be aware that the claims that weight on a wheel is more important than weight on the frame is based on an evaluation of weight at the perimeter of the wheel, not at the center so the weight savings of the hub may not be as critical as weight savings elsewhere in the wheel.

Aerodynamics-wise, low spoke count wheels are better.


  Fewer spokes means decreased stiffness and spoke life, while additional spoke 
  counts can lead to higher power requirements, or overly stiff wheels which are punishing on rough roads.


Low spoke count wheels are not necessarily lighter though -- I believe most low spoke count wheels use heavier gauge spokes and nipples to compensate for the higher tension -- the bigger nipple spreads the force over a larger area of the rim to prevent light rims from buckling.



(Leaving the rest here so this answer is useful to other readers)

It depends what you're going to use the wheel for, but I'm answering assuming you want a reasonably light durable wheel that you can fix yourself if you have a breakdown, and that you don't have a van pacing you with spare wheels.


  What's the most important part of the wheel? Hub or rim? Or both?


Spokes.  A low spoke count wheel on a high quality rim and hubs is still a crap wheel.  Low spoke count wheels require each spoke to be that much more highly tensioned.  Low spoke count wheels are made by machines that push on the whole perimeter of the wheel as they tension the spokes.  No human can do that on the side of the road, so this means that if one spoke breaks, you have no real chance to retrue the wheel by tightening or loosening other spokes to compensate.

High spoke count wheels (36 spokes for example) may have slightly higher drag, but there's so many spokes, that you can actually use that little spoke wrench that comes with your multi-tool to do road-side repairs.


  How do you evaluate or compare different hubs?


Look at reviews.  You basically have to relace the wheel every time you go through a hub, so I prefer long-life over lighter-weight, buy YMMV.

Phil wood is my personal favorite, but again, probably not ideal for racing.


  How do you evaluate or compare different rims/spoke combinations?


Jobst Brandt's book "The Bicycle Wheel" explains spoke lacing patterns.

Major rim choices include


offset vs centered spoke holes.  You can get less dish (see below) sometimes by using offset spoke holes
depth of rims
thickness of material.  Thinner is lighter, but if you're going to do a lot of wet weather riding and you have rim-brakes, then grit gets into the brake pad which will wear down thinner rims more quickly than thicker rims.
material.  Some people actually sell Carbon Fiber rims but if you're using rim brakes then you want the rim to be made of a heat-conductive material so it sheds heat well.  Carbon fiber does not.
type of tire.  Clinchers vs tubular.  If you don't know what this means, then you're using clincher tires which have a metal (or kevlar) bead, a stiff wire that runs around the inner perimeter of the tire and presses it against the rim.



  Are there other factors I've missed?


Dish.  If you have a derailleur on one side of the wheel, you can't quite center the wheel over the hub so you have to use longer spokes on one side.  This means you can't get both sets of spokes up to the same tension you could if you didn't dish the wheel resulting in a slightly weaker wheel.