iMac inconsistenly hanging and showing beach ball
Your own advice is fairly good in this scenario. 
There is a possibility that you have just some serious file system corruption, but usually Disk Utility should catch that. If you want, you can boot into single user mode (CMD S on boot) and it will give you a little file system cleanup command you can run
fsck -fy and then press enter, when it finishes you reboot. 

If this doesn't help, reinstalling after performing a backup is not a bad idea because it will isolate it to a software or hardware issue. The hard drive going over some area repeatedly could be just file system screw ups, but usually repeated noises are a bad sign for a hard drive, clicking etc. 

If the problem comes back after the reinstall you definitely want to look into a new hard drive. Four years is not an uncommon life span for some hard drives.

Also just a quick point of clarification, the S.M.A.R.T status doesn't predict all failures, it just presages specific ones it can determine. Just because it isn't telling you your drive is failing doesn't mean it wont. Keeping everything backed up is a really good idea at this point.