Python Multithreaded Messenger Simulation. Stuck on timerThread update. What do?
Question: I have a piece of code that simulates a system of messengers (think post
office or courier service) delivering letters in a multithreaded way. I want
to add a way to manage my messengers "in the field" to increase the efficiency
of my system.

tl;dr: How do I update my tens-to-hundreds of timerthreads so they wait longer
before calling their function?

Here's what the code I've written so far is supposed to do in steps.

  1. Someone asks for a letter

  2. We check to see if there are any available messengers. If none, we say "oops, sorry. can't help you with that"

  3. If at least one is available, we send the messenger to deliver the letter (new timer thread with its wait param as the time it takes to get there and back)
  4. When the messenger gets back, we put him in the back of the line of available messengers to wait for the next delivery

I do this by removing Messenger objects from a double ended queue, and then
adding them back in after a timerthread is done waiting. This is because my
Messengers are all unique and eventually I want to track how many deliveries
each has had, how far they have traveled, and other stuff.

Here's a pseudoish-codesnippet of the larger program I wrote for this

    
    
    numMessengers=5                                        
    messengerDeque=deque()                             
    pOrder=0.0001
    
    class Messenger:
        def __init__(self):
    
    for i in range(numMessengers):
        messenger=Messenger()
        messengerDeque.append(messenger)
    
    def popDeque():
        messenger=idleDeque.popleft()
        print 'messenger #?, sent'
        return messenger
    
    def appendDeque(messenger):
        print 'messenger #?, returned'
        messengerDeque.append(messenger)
    
    def randomDelivery():
        if numpy.random.randint(0,10000)<=(pOrder*10000):
            if len(messengerDeque)!=0:
                messenger=popDeque()
                tripTime=distance/speed*120
                t=threading.Timer(tripTime,appendDeque,args=[messenger])
                t.start()
            else:
                print "oops, sorry. can't help you with that"
    

The above works in my program.

What I would like to add is some way to 'reroute' my messengers with new
orders.

Lets say you have to deliver a letter within an hour of when you get it. You
have five messengers and five orders, so they're all busy. You then get a
sixth order. Messenger 2 will be back in 20 minutes, and order six will take
30 minutes to get to the delivery destination. So instead of saying "oops, we
can't help you". We would say, ok, Messenger 2, when you get back, immediately
go deliver letter six.

With the code I've written, I think this could be done by checking the active
threads to see how long until they call their functions, pick the first one
you see where that time + how long your new delivery takes is < 1 hr, cancel
it, and start a new thread with the time left plus the new time to wait.

I just don't know how to do that.

How do you check how long is left in a timerthread and update it without
making a huge mess of your threads?

I'm also open to other, smarter ways of doing what I described.

YAY PYTHON MULTITHREADING!!!!! Thanks for the help


Answer: Using the class threading.Timer wont fulfill your needs. Although there is a
"interval" member in Timer instances, once the Timer(thread) started running
any changes in interval (time-out) are not considered. Furthermore you need to
know how much time is still left for the timer to be triggered, for which
there isn't a method as far as I know. Furthermore you probably also need a
way to identify which Timer instance you need to update with the new timeout
value, but this is up-to you.

You should implement your own Timer class, perhaps something along the lines
of:

    
    
    import threading
    import time
    
    class MyTimer(threading.Thread):
        def __init__(self, timeout, event):
            super(MyTimer, self).__init__()
            self.to = timeout
            self.evt = event
    
        def setTimeout(self, v):
            self.end = time.time() + v
    
        def run(self):
            self.start = time.time()
            self.end = time.time() + self.to
            while self.end > time.time():
                time.sleep(0) # instead of thread.yield
            self.evt()
    
        def getRemaining(self):
            return self.end - time.time()
    
    def hi(): print "hi"
    
    
    T=MyTimer(20,hi)
    T.start()
    for i in range(10):
        time.sleep(1)
        # isAlive gives you True if the thread is running
        print T.getRemaining(), T.isAlive() 
    
    T.setTimeout(1)
    for i in range(3):
        time.sleep(1)
        print T.getRemaining(), T.isAlive()
    

