Should I restart after uninstalling software or drivers on a mac?
I'm going to just posit a general rule of thumb here - 

If it needs you to restart it will tell you - or even make you.

More specifically,
If it doesn't need an uninstaller, then just trashing the app requires no further action.
If you use something like AppCleaner to also sweep up the prefs, there is still no further action required.

If it needs an uninstaller, the uninstaller will tell you if a restart will be required.

However, just to be on the safe side...
If you manually pull kexts etc out of the system using Terminal - restart is safest.

The rule of thumb works on a simple 'Did it change the system?'
If not, then no restart
If it did, then yes.