In England, can I buy fresh fish directly off the boat?
People don't become fishermen in the UK to get rich. We simply don't eat enough variety of fish for them to sell it in person. The way they make the most money is wholesaling it for export to the rest of the EU. 

That said, apart from possibly Yarmouth you are in one of the best areas in the country where you might get this service. Here's one based around Cornwall:

http://www.mouseholefish.com

They put posters up when they will be heading out. 

For more suggestions you might consider a Facebook group or similar for your town?