Why do builders say that wet concrete/mortar is not as strong?
My only guess is that a watery mix will have the layers settle in
  their different sedimentary layers with the fine cement settling to
  the bottom...


One reason is that the aggregate can settle to the bottom much too easily, yes. This is called segregation and results in non-cohesiveness of concrete.
Another reason too much water is bad is that it can result in porous concrete, especially at the surface.


  "Concrete that is poured too wet will be weak regardless of how it is
  cured. Not all of the water that is added to the concrete mix is
  necessary for the hydration process. In fact, the amount of water
  required to completely hydrate the cement is only about one half to
  two thirds of what is usually added at the batch plant or on the job
  site. The rest is added strictly to make the mix more workable. Adding
  too much water, however, might save work during placement, but it will
  also result in very weak, porous concrete, even with proper curing." - source


Excerpt from a good article (read the whole article) describing this problem in good detail:


  "When there is too much water in the concrete, there is greater
  shrinkage with the possibility for more cracks and reduced compressive
  strength. As a general rule, every additional inch of slump decreases
  strength by approximately 500 psi. So for example, if you ordered
  5-inch slump concrete and received 7½ inches, a mix designed to be
  4000 psi would end up being 2500 psi. This represents a serious loss
  in strength, especially if you were placing exterior concrete in a
  freeze/thaw climate where the standard requirement is 4000 psi for
  proper durability." - source


This is often the reason for the slump tests that are done on most large projects. It tests the workability, but also indirectly tests if there is too much water.