Does FileVault hurt responsiveness on a spinning HDD?
Once the initial encryption has finished you are very unlikely to notice any difference in day-to-day usage.

Your iMac's CPU also contains specific features that assist with encryption (AES).