Resolving DNS in bulk
<p>I need to resolve a large number (hundreds of thousands) of domains to IP addresses in Java. While using <code>InetAddress.getByName()</code> is feasible for small numbers it is far to slow for use in large quantities (probably because it is only sending one request at a time to the DNS server and waiting for the response before moving on to the next one).</p>

<p>Is there a more efficient way (such as sending them to the DNS server in bulk) that would cut down the time required to resolve a large number of domains?</p>

<p>At fmucar's request I'm adding the code used to try a more multi-threaded approach:</p>

<pre><code>Set&lt;String&gt; ips = Collections.synchronizedSet(new HashSet&lt;String&gt;());
int i = 0;
List&lt;Set&lt;String&gt;&gt; sets = new ArrayList&lt;Set&lt;String&gt;&gt;();
for (String host : domains) {
    if (i++ % 5 == 0) {
        sets.add(new HashSet&lt;String&gt;());
    }
    Set&lt;String&gt; ipset = sets.get(sets.size()-1);
    ipset.add(host);
}
for (Set&lt;String&gt; ipset : sets) {
    Thread t = new Thread(new DomainResolver(ips, ipset));
    t.start();
}
</code></pre>

<p>At 250 per thread we peaked around 700 results per minute. Which, while better than before (&lt;300) was still not that great when needing to resolve hundreds of thousands. Lowering it to only 5 per thread greatly speeds this up to several thousand per minute. This obviously creates an insane amount of threads though, so presently investigating doing the resolving in C to make use of <a href="http://www.chiark.greenend.org.uk/~ian/adns/" rel="nofollow">http://www.chiark.greenend.org.uk/~ian/adns/</a> </p>
