Postfix SMTP server slow to accept incoming connection
The Delay is due to postfix doing a reverse lookup on the client IP and doing another lookup on that result to see if they get back to the IP.

if you are using a consumer connection to the internet most likely the first test will pass. but the second test will cause the problem.

you can verify this on the server itself.

nslookup &lt;your client IP&gt;


returns something like 80-41-90-213.dsl.iwx.uk.net

but on reversing that

nslookup 80-41-90-213.dsl.iwx.uk.net 


you will probably get SERVFAIL after a delay.

it is that delay that is showing up when you connect to the server.
To find out which one go through each server in /etc/resolve.conf

nameserver x.x.x.x
nameserver y.y.y.y
nameserver z.z.z.z


test them with

nslookup 80-41-90-213.dsl.iwx.uk.net x.x.x.x


One or more will show a delay before they come back to the command prompt.

Once you've found which DNS causes the big SERVFAIL delay, replace it with another. i.e. Googles 8.8.8.8 and 8.8.4.4 currently return relatively quickly.