Rigid or Flexible bathroom exhaust duct
Rigid duct is absolutely better--it's much less likely to accumulate lint and clog or create a fire hazard. However, I wouldn't be concerned about using a short length of semi-rigid (metal) duct at the fan end to allow for some install flexibility.
You could also make all the connections and rely on some pivot at the opposite end of the initial straight run to accommodate height variation. I can't imaging your builder not being able to work with that.