Is Yosemite Mail caching signature files? How can I get around that?
I had the same problem as you. I did some digging around and found a solution that used to work back in the OSX 10.8 days and stopped working with OSX 10.9. Oddly enough, it works again with OSX 10.10.1 and Mail 8.1 (1993).

There are multiple parts to this - an AppleScript that you compile into an Application, a text file containing your random quotes, two signatures in Mail.app (one is a template, the other is the actual signature for your messages), and the system 'cron' service. (Ideally you should use launchd, but I haven't got that far yet.)

The AppleScript:

if application "Mail" is running then
    set AppleScript's text item delimiters to "~###~"
    set myQuote to some text item of (read "/Users/Steve/.sigs.in")
    tell application "Mail"
       set the content of signature "MySignature" to the content of signature "MyTemplate" &amp; myQuote
    end tell
end if


In my implementation (which you can deduce from the AppleScript):


My two Mail.app signatures are called "MyTemplate" and "MySignature".
"MyTemplate" contains everything that is static - stuff like my email
address, etc. "MySignature" is the one that I use with my messages.
The text file holding my quotes is called ".sigs.in"
(so that it doesn't appear in Finder windows) and all the quotes are
separated by a line holding only "~###~". This file lives in my home directory.
My crontab entry consists
of * * * * * /Users/Steve/bin/MailRandomSig.app/Contents/MacOS/applet 


The compiled AppleScript lives in /Users/Steve/bin (obviously enough), and the "applet" part is the actual compiled binary that we want to run.

I've tested this and it works beautifully. Each minute, cron kicks off my compiled script which updates the signature.

The only "downside" is that if you don't like the quote you have to wait a minute for the script to refresh the signature and then reselect the signature from the drop-down selector in the message composition window. (I have two signatures - one static, the other the output of my script - to get around this.)