Why can't aircraft be cleared to land before the preceding aircraft turns off the runway?
Actually, the FAA doesn't always require the preceding airplane to clear the runway if they have the required separation unless it is dark (when it is very hard to visually estimate distances).

According to the FAA Air Traffic Control Order Section 10. Arrival Procedures and Separation:


  3-10-6. ANTICIPATING SEPARATION
  
  a. Landing clearance to succeeding aircraft in a landing sequence need
  not be withheld if you observe the positions of the aircraft and
  determine that prescribed runway separation will exist when the
  aircraft crosses the landing threshold. Issue traffic information to
  the succeeding aircraft if a preceding arrival has not been previously
  reported and when traffic will be departing prior to their arrival.
  
  EXAMPLE- “American Two Forty-Five, Runway One-Eight, cleared to land,
  number two following a United Seven-Thirty-Seven two mile final.
  Traffic will depart prior to your arrival.”
  
  “American Two Forty-Five, Runway One-Eight, cleared to land. Traffic
  will depart prior to your arrival.”
  
  NOTE- Landing sequence number is optional at tower facilities where
  the arrival sequence to the runway is established by the approach
  control.
  
  b. Anticipating separation must not be applied when conducting LUAW
  operations, except as authorized in paragraph 3-10-5b2. Issue
  applicable traffic information when using this provision.
  
  EXAMPLE- “American Two Forty-Five, Runway One-Eight, cleared to land.
  Traffic will be a Boeing Seven-Fifty-Seven holding in position.”


And the separation is specified in the same document:


  3-10-3. SAME RUNWAY SEPARATION
  
  a. Separate an arriving aircraft from another aircraft using the same
  runway by ensuring that the arriving aircraft does not cross the
  landing threshold until one of the following conditions exists or
  unless authorized in para 3-10-10, Altitude Restricted Low Approach.
  
  
  The other aircraft has landed and is clear of the runway. (See FIG 3-10-1.) Between sunrise and sunset, if you can determine distances by
  reference to suitable landmarks and the other aircraft has landed, it
  need not be clear of the runway if the following minimum distance from
  the landing threshold exists:
  
  
  (a) When a Category I aircraft is landing behind a Category I or II-
  3,000 feet.
  
  (b) When a Category II aircraft is landing behind a Category I or II-
  4,500 feet.