Server response time high when InnoDB tables active but not being accessed
[SOLVED]

I enabled query_cache_type=1 and query_cache_size=256M. 

The problem went away and the server response reduced from 6 seconds to 1 second. Very strange considering most recommend that Query cache should be disabled!