Why were the speed records set by seaplanes in the early thirties?
The answer: Fixed-pitch propellers.

Until the 1930s propellers were fixed pitch only. From Wikipedia:


  The first practical controllable-pitch propeller for aircraft was
  introduced in 1932. French firm Ratier pioneered variable-pitch
  propellers of various designs from 1928 onwards, relying on a special
  ball bearing helicoïdal ramp at the root of the blades for easy
  operation.


This means all speed records were flown with fixed-pitch propellers, which needed to be optimised to the top design speed to coax as much thrust from the engines as possible. In turn, this means that most of the propeller blade was stalled at low speed, so take-off thrust was low. To achieve high speed, a high wing loading was needed and, in combination with low thrust, the take-off length of such an aircraft was beyond of the length of airfields back then.

On water, however, the maximum field length is close to infinity. Yes, drag with floats is especially high when transitioning from buoyancy to dynamic lift (the "hump"), but that can be overcome when that hump occurs at high enough speed so the propeller delivers enough thrust.