Mysql database slow responding to connects when having only few concurrent connections
The problem was that we have a haproxy in front of the setup. Each call to a webserver was creating a dns query on each of the webservers because the virtualhost was a "servername" with no ip supplied. Adding a local dns bind caching daemon solved this issue although its not the best sollution. (also didn't show up in the beginning since it was udp traffic)

So in the end this was not database related.