Making sense of enzyme Km comparisons
Since the Michaelis-Menton constant Km is the concentration of substrate at 0.5Vmax, it is an inverse measure of its substrate affinity, because a lower Km indicates that less substrate is needed to reach a certain reaction speed. Hence, a low Km means a high substrate affinity. 

Your statement


  "Maybe the maximum velocity (Vmax) of higher-Km enzymes is higher? Then of course Km can have a higher value." 


is incorrect. Km characterizes how steep reaction speed increases with substrate availability; it does not determine maximum speed.

Lastly, to address your title question, comparing affinities can make a lot of sense. For example, consider the case where enzymes catalyzing similar reactions in different species of organisms are compared. Very low Km means optimal use of small substrate levels, while a high Vmax shows optimized reaction speeds. This in turn may tell you something about optimal habitats and evolutionary pressure.