Ceiling fan light flickers whenever fan is turned off
Two gotchas here. 


You can't put a dimmer on a fan.  Fans require a fan speed controller which is a different sort of thing.  Even if you never use the dimmer in middle settings, most dimmers are also doing that dimming thing at lowest (nominal off) and highest settings.  Since you don't care that much about dimming, change it to a plain switch.  
You must not put a fan on a plain junction box.  It doesn't have the strength.  A fan's vibration will tear a normal junction box out of the wall. So change that box to a fan-rated junction box if it isn't one already.