Python and ADNS, falling in infinite loop somewhere
Question: I have written some code that queries adns. Problem with this code is that it
gets stuck, how? Let me explain it:

  * Say my dnslist is ["8.8.4.4", "8.8.8.8", "208.67.220.220", "208.67.222.222", "192.168.50.1"] 
  * It would pop a dns from the list and query againt it, now that means that DNS will be queried in reverse order
  * No matter what i do, It never shows results from the dns it picked up first (in our case 192.168.50.1)
  * I was not sure if that dns ever replied so 
    * First i changed DNS list to contain just that last DNS Server and code executes fine
    * Second i used the old list with 5 DNS servers except that the last one was managed by my so i could track if code even queries it or not, and to my surprise the query does take place.
    * So query is made, we get result but that result is never inserted into resolved_hosts for some reason, and because that results is not inserted, its length will remain less than the length of dnslist, causing a infinite loop.

What do you think could be causing this problem and how to solve it?

# Code Execution Results

    
    
    Inside class's init'
    Data
    host www.yahoo.com
    dnslist length 5
    intensity 1
    Inside resolve()
    inside finished_resolving()
    Resolved : 0/5
    Inside 'while not finished_resolving'
    Queue: 0/1
    Launching Querying for www.yahoo.com/1 on 192.168.50.1
    Queue: 1/1
    Launching Querying for www.yahoo.com/1 on 208.67.222.222
    inside collect_results()
    inside finished_resolving()
    Resolved : 0/5
    Inside 'while not finished_resolving'
    ------------------------ CLIPPED ----------------
    Inside 'while not finished_resolving'
    inside collect_results()
    Inside collect_results's for query in self.adns.completed()
    DNS used was208.67.222.222
    Answered : (0, 'any-fp.wa1.b.yahoo.com', 1286169807, ('69.147.125.65', '67.195.160.76'))
    Resolved www.yahoo.com to 69.147.125.65 using 208.67.222.222
    Resolved hosts count1
    And they are: 
    {'208.67.222.222': '69.147.125.65'}
    
    
    inside finished_resolving()
    Resolved : 1/5
    Inside 'while not finished_resolving'
    Queue: 1/1
    Launching Querying for www.yahoo.com/1 on 208.67.220.220
    inside collect_results()
    inside finished_resolving()
    Resolved : 1/5
    Inside 'while not finished_resolving'
    -------------------------- CLIPPED --------------------
    inside collect_results()
    Inside collect_results's for query in self.adns.completed()
    DNS used was208.67.220.220
    Answered : (0, 'any-fp.wa1.b.yahoo.com', 1286169790, ('67.195.160.76', '69.147.125.65'))
    Resolved www.yahoo.com to 67.195.160.76 using 208.67.220.220
    Resolved hosts count2
    And they are: 
    {'208.67.222.222': '69.147.125.65', '208.67.220.220': '67.195.160.76'}
    
    
    inside finished_resolving()
    Resolved : 2/5
    Inside 'while not finished_resolving'
    Queue: 1/1
    Launching Querying for www.yahoo.com/1 on 8.8.8.8
    inside collect_results()
    inside finished_resolving()
    Resolved : 2/5
    Inside 'while not finished_resolving'
    -------------------------- CLIPPED --------------------
    inside collect_results()
    Inside collect_results's for query in self.adns.completed()
    DNS used was8.8.8.8
    Answered : (0, 'eu-fp.wa1.b.yahoo.com', 1286169758, ('87.248.122.122',))
    Resolved www.yahoo.com to 87.248.122.122 using 8.8.8.8
    Resolved hosts count3
    And they are: 
    {'208.67.222.222': '69.147.125.65', '208.67.220.220': '67.195.160.76', '8.8.8.8': '87.248.122.122'}
    
    
    inside finished_resolving()
    Resolved : 3/5
    Inside 'while not finished_resolving'
    Queue: 1/1
    Launching Querying for www.yahoo.com/1 on 8.8.4.4
    inside collect_results()
    inside finished_resolving()
    Resolved : 3/5
    Inside 'while not finished_resolving'
    -------------------- CLIPPED -------------------------------------
    inside collect_results()
    Inside collect_results's for query in self.adns.completed()
    DNS used was8.8.4.4
    Answered : (0, 'eu-fp.wa1.b.yahoo.com', 1286169757, ('87.248.122.122',))
    Resolved www.yahoo.com to 87.248.122.122 using 8.8.4.4
    Resolved hosts count4
    And they are: 
    {'208.67.222.222': '69.147.125.65', '208.67.220.220': '67.195.160.76', '8.8.8.8': '87.248.122.122', '8.8.4.4': '87.248.122.122'}
    
    
    inside finished_resolving()
    Resolved : 4/5
    Inside 'while not finished_resolving'
    inside collect_results()
    inside finished_resolving()
    Resolved : 4/5
    ---------------- CLIPPED -------------------------------
    

(last block keeps repeating until system starts to hang up, load goes upto 24)

# Code

**test.py**

    
    
    import adns
    from time import time
    from async_dns import AsyncResolver
    
    
    
    dnslist2 = ["8.8.4.4", "8.8.8.8", "208.67.220.220", "208.67.222.222", "192.168.50.1"]  #192.168.50.1  is a dns server i manage
    host = "www.yahoo.com"
    record = adns.rr.A
    intensity = len(dnslist2)/5+1
    
    ar = AsyncResolver(dnslist2, host, record, intensity)
    start = time()
    resolved = ar.resolve()
    end = time()
    
    print "\n\n"
    
    for dns, ip in resolved.items():
      if ip is None:
        print "%s could not resolv %s." % (dns, host)
      else:
        print "%s resolved it to %s : %s" % (dns, host, ip)
    
    print "\n\n----------------------------------------------------"
    print "It took %.2f seconds to query %d dns." % (end-start, len(dnslist))
    print "----------------------------------------------------"
    

**async_dns.py**

    
    
    #!/usr/bin/python
    #
    
    import sys
    import adns
    from time import time
    
    class AsyncResolver(object):
        def __init__(self, dnslist, host, record, intensity=10):
    
            """
            dnslist: a list of dns used to resolve
            host : hostname to resolve
            record: recordtype to resolve
            intensity: how many hosts to resolve at once
            """
            print "Inside class's init'"
            self.dnslist = dnslist
            self.host = host
            self.record = record
    
    
            if intensity >= len(dnslist) :
                self.intensity = len(dnslist)/5+1
            else:
                self.intensity = intensity
    
            print "Data"
            print "host " + host
            print "dnslist length " + str(len(dnslist))
            print "intensity " + str(intensity)
    
    
    
    
        def resolve(self):
            """ Resolves hosts and returns a dictionary of { 'dns': 'ip' }. """
            print "Inside resolve()"
    
            host = self.host
            record = self.record
            resolved_hosts = {}
            active_queries = {}
            dns_queue = self.dnslist[:]
    
            def collect_results():
                print "inside collect_results()"
                for query in self.adns.completed():
                    print "Inside collect_results's for query in self.adns.completed()"
                    answer = query.check()
                    dns = active_queries[query]
                    print "DNS used was" + dns
                    print "Answered : " + str(answer)
                    del active_queries[query]
                    if answer[0] == 0:
                        #print "Inside answer[0] == 0 , ip:" + answer[3][0]
                        ip = answer[3][0]
                        resolved_hosts[dns] = ip
                        print "Resolved %s to %s using %s" % (host, ip, dns)
                        print "Resolved hosts count" + str(len(resolved_hosts))
                        print "And they are: "
                        print str(resolved_hosts)
                        print "\n"
    
                    elif answer[0] == 101 and not record == adns.rr.CNAME: # CNAME if CNAME wasn't required'
                        print "ooopppps, i got a CNAME, gotta find its A"
                        print "\n"
                        query = self.adns.submit(answer[1], adns.rr.A)
                        active_queries[query] = dns
                    else:
                        resolved_hosts[dns] = None
                        print "THIS COULD NOT BE RESOLVED"
    
            def finished_resolving():
                print "inside finished_resolving()"
                print "Resolved : " + str(len(resolved_hosts)) + "/" + str(len(self.dnslist))
                return len(resolved_hosts) == len(self.dnslist)
    
            while not finished_resolving():
                print "Inside 'while not finished_resolving'"
                while dns_queue and len(active_queries) <= self.intensity:
                    print "Queue: " + str(len(active_queries)) + "/" + str(self.intensity)
                    dns = dns_queue.pop()
                    self.adns = adns.init(adns.iflags.noautosys,sys.stderr,"nameserver "+dns)
                    query = self.adns.submit(host, record)
                    print "Launching Querying for " + host + "/" + str(record) + " on " + dns
                    active_queries[query] = dns
                collect_results()
            return resolved_hosts
    


Answer: I have just attempted to use adns python binding and have experienced a
similar problem - the completed queries array never gets completely filled,
causing the check loop to run forever. In my case, it appeared that domains
that could not be resolved (due to NXDOMAIN etc) were never added to the
completed queries array (with empty/Null value)

I had a look at the C binding but could not see a separate data structure for
holding failed results, so it does appear that these should be added to the
'completed' queries somehow. I suspect there is a bug in the binding or adns
library preventing this from happening.

