Is it illegal for Joe Public to listen to ATC in the UK?
Yes, it is illegal.  

The UK has some very strange ideas about Radio, which bear no resemblance to the reality of physics and how radio actually works. Foremost among those ideas is the idea that you are only legally allowed to listen to transmissions intended for "general reception":


  The services that can be listened to under the definition of general reception are:
  
  
  licensed broadcasting stations;
  amateur and citizens' band radio transmissions; and
  weather and navigation transmissions
  
  
  Radio scanners should not be used to listen to any other radio services, including illegal radio stations (pirates) (by virtue of the fact that they are not licensed radio stations).


Aviation-band transmissions are not on Ofcom's list, so even though they are broadcast entirely "in the clear" and anyone with an appropriately tuned AM receiver could hear them you are legally prohibited from listening to them.



Providing feeds to LiveATC would also appear to be a violation, though it's not clear to me that listening to LiveATC streams would put the listener in violation of the law.

From the same Ofcom site:


  … it is illegal to listen to anything other than general reception transmissions unless you are either a licensed user of the frequencies in question or have been specifically authorised to do so by a designated person.


and 


  … it is also illegal to tell a third party what has been heard in a transmission a person has listened to illegally.


So anyone providing UK aviation band transmissions to LiveATC may be in violation of both of those laws, but I'm not certain what that means for LiveATC listeners (that would be a question to ask Ofcom).