What are the rules in the UK about overflying built-up areas for fun?
Does anyone know what the rules are for just randomly deciding to fly
  over a built-up area like this for the fun of it?


In UK, the master document for air regulations is The Rules of the Air Regulations (ROTAR). UK being still part of the EU, they refer to a common set of rules at the EU level, the Standardised European Rules of the Air aka SERA (qui sera sera...)

UK ROTAR state a minimum flyover height, one for VFR, one for IFR:


  The minimum heights for VFR flights shall be those specified in
  SERA.5005(f) and minimum levels for IFR flights shall be those
  specified in SERA.5015(b)


and SERA says:


  SERA.5005:  
  
  (f) Except  when  necessary  for  take-off  or  landing,  or  except 
  by  permission  from  the  competent  authority,  a  VFR  flight shall 
  not  be  flown:
  
  
  (1) over  the  congested  areas  of  cities,  towns  or  settlements
  or  over  an  open-air  assembly  of  persons  at  a height  less 
  than  300  m  (1  000  ft)  above  the  highest obstacle  within  a 
  radius  of  600  m  from  the  aircraft;
  (2) elsewhere  than  as  specified  in  (1),  at  a  height  less 
  than 150  m  (500  ft)  above  the  ground  or  water,  or  150  m 
  (500  ft)  above  the  highest  obstacle  within  a  radius of 150 m 
  (500  ft)  from  the  aircraft.
  


So for VFR: 150 m, but 300 m in populated areas.


  SERA.5015:
  
  
  (2) elsewhere  than  as  specified  in  (1),  at  a  level  which 
  is  at  least  300  m  (1  000  ft)  above  the  highest  obstacle 
  located within  8  km  of  the  estimated  position  of  the  aircraft.
  


And for IFR: 300 m above obstacles.

As you see this is not very high. In addition, as you mentioned, aircraft landing or taking off will indeed fly below these minimums.


  Jets landing at LHR with a stupidly shallow glide path angle


Landing cannot be done at high angle, the proper slope is the result of two elements:


Aircraft cannot afford a high descent speed, since the vertical rate determines how hard the landing gears will touch the runway and whether they will be damaged.
Aircraft cannot slow down under a given speed without losing their capability to create lift. This minimum speed and the previous maximum vertical speed determine a maximum slope.


This glide slope has been chosen so that it is compatible with all sorts of aircraft approach speeds and vertical rates.

A standardized value of 3° is common. But when the approach area is limited, or when this is desirable to maintain a higher altitude a longer time to reduce noise on the ground, then this angle can be increased to 3.5 or 4.5°, or even 5.5°, this is the case for some approaches at Heathrow as @TomMcW told us.


Source

Alternatively, some airports have lower angles, down to 2.5°.