Should central heating pipes through internal brickwork have insulation?
No disrespect to ChrisF but I disagree fully.

4" of common brick has an R-Value of only 0.88 - so its a pretty good conductor of heat, which isn't what you want when you want to conserve heat.  The direct contact with the wall will only improve heat conduction, as well.  

UNFORTUNATELY - given the tight space, you aren't going to be able to insulate where you need it most, the point of direct contact.

If you're not averse to a little more elbow grease - your 4" pipe has a 12.5 sq in internal surface area (ie: the area of the circle), so you could replace the round 4" pipe with either fairly small square duct (if you can find it, 3x4 is pretty small) or insulated round pipe.  If you go with the square duct, you can go larger than necessary (just don't go smaller) and box it in with Styrofoam board cut to fit to give a good thermal break.