Does having more enabled modules make the site slower?
Yes.
However, 30 is nothing to worry about. You're in trouble when you go over 100 ;)

Install a bytecode cache (APC or XCache) to eliminate most of the slowdown of including many files, and make sure to periodically remove unused modules (usually every project gathers modules that get unused at some point during development)