Do I need a heat sink for small voltage regulator?
Do the math.  The regulator will drop 4 V with 10 mA thru it.  (4 V)(10 mA) = 40 mW.  That's not much at all, and even a small package like a SOT-23 can safely dissipate that when mounted normally on a board.

However, you mentioned this will be encased.  That changes the thermal resistance from junctions to ambient.  Ultimately its about keeping the junctions below some maximum temperature.

A typical junction to ambient resistance for a SOT-23 is about 360 &deg;C/W.  At 40 mW that means the junction would be 15&deg;C above ambient.  If the part can operate at up to 120&deg;C junction temperature, for example, and ambient is 25&deg;C, then the device could dissipate up to 260 mW.

The calculations are the same with a potted circuit, except that the junction to ambient thermal resistance will be different.  It can be both higher or lower, depending on the nature of the potting compound and its depth.  This is something you have to look into by carefully examining the potting compound datasheet, and possibly talking to the manufacturer and maybe running your own experiments.

One strategy is to grossly over-spec it.  For example, if you get a 7808 in a TO-220 package, then it's hard to imagine there will be a problem except for some extreme potting situations.