Buttermilk Fudge not setting
I would bet that the "something" your grandma said about shine was "beat it until it's not shiny anymore."  The likely reasons that your fudge isn't setting are that you're either not heating it hot enough to begin with, not waiting long enough before you start beating, or you're not beating it until it loses its shine.  

Here's how I'd re-write your recipe's instructions to make them a bit more detailed.

Version 1 (you don't own a candy thermometer):
Heat the first 5 ingredients until medium soft ball stage.
Remove from heat and cool without disturbance for about 10-15 minutes. 
Add vanilla and nuts beat until the candy begins to lose its shine.
When it is beginning to thicken and becomes opaque, pour/scrape it into a prepared pan.

Version 2 (you own a candy thermometer):
Heat the first 5 ingredients until 240 F.
Remove from heat and cool without disturbance until candy reaches 140 F.
Add vanilla and nuts beat until the candy begins to lose its shine.
When it is beginning to thicken and becomes opaque, pour/scrape it into a prepared pan.