Bottle causing a python program to crash? Uses simple implementation of threads, queues, and forking
<p>I've tried to simplify this as much as possible but I'm still getting an error. I have a simple http server (bottle) that upon receiving a post request executes a function which is supposed to quickly fork itself. The parent process simply returns a job ID and closes while the child process continues to process same data (which is a list of URLs). I've removed all the input and output functions and hard coded the data but my program is still crashing. The funny part is when I alter the program to run directly at the command line rather then start an http server and wait for bottle to execute it everything works fine!</p>

<pre><code>#!/usr/bin/python
#This is a comment
import sys, time, bottle, os
from threading import Thread
from Queue import Queue
from bottle import route, run, request, abort

num_fetch_threads = 2
url_queue = Queue()

def fetchURLContent(i, q):
  while True:
    #print '%s: Looking for URLs in queue' % i
    url = q.get()
    #print 'URL found: %s' % url[0]
    q.task_done()
    time.sleep(1)

@route('/', method='POST') # or @route('/login', method='POST')
def main():

  urls = ['http://www.yahoo.com', 'http://www.google.com']

  newpid = os.fork()
  if newpid == 0:

    for i in range(num_fetch_threads):
      worker = Thread(target=fetchURLContent, args=(i, url_queue))
      worker.setDaemon(True)
      worker.start()
    print 'Queuing: ', url
    for url in urls:
      url_queue.put(url)
    time.sleep(2)
    print 'main thread waiting...'
    url_queue.join()
    print 'Done'

  else:
    print "Your job id is 5"
    return
def webServer():
  run(host='33.33.33.10', port=8080)

if __name__ == "__main__":
  print 'Listening on 8080...'
  webServer()
</code></pre>

<p>The error message I get is as follows:</p>

<pre><code>Listening on 8080...
Bottle v0.11.3 server starting up (using WSGIRefServer())...
Listening on http://33.33.33.10:8080/
Hit Ctrl-C to quit.

33.33.33.1 - - [19/Oct/2012 21:21:24] "POST / HTTP/1.1" 200 0
Traceback (most recent call last):
  File "/usr/lib/python2.7/wsgiref/handlers.py", line 86, in run
    self.finish_response()
  File "/usr/lib/python2.7/wsgiref/handlers.py", line 128, in finish_response
    self.finish_content()
  File "/usr/lib/python2.7/wsgiref/handlers.py", line 246, in finish_content
    self.send_headers()
  9 url_queue = Queue()
  File "/usr/lib/python2.7/wsgiref/handlers.py", line 268, in send_headers
    self.send_preamble()
  File "/usr/lib/python2.7/wsgiref/handlers.py", line 189, in send_preamble
    self._write('HTTP/%s %s\r\n' % (self.http_version,self.status))
  File "/usr/lib/python2.7/wsgiref/handlers.py", line 389, in _write
    self.stdout.write(data)
  File "/usr/lib/python2.7/socket.py", line 324, in write
    self.flush()
  File "/usr/lib/python2.7/socket.py", line 303, in flush
    self._sock.sendall(view[write_offset:write_offset+buffer_size])
error: [Errno 32] Broken pipe
----------------------------------------
Exception happened during processing of request from ('33.33.33.1', 57615)
Traceback (most recent call last):
  File "/usr/lib/python2.7/SocketServer.py", line 284, in _handle_request_noblock
    self.process_request(request, client_address)
  File "/usr/lib/python2.7/SocketServer.py", line 310, in process_request
    self.finish_request(request, client_address)
  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__
    self.finish()
  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish
    self.wfile.flush()
  File "/usr/lib/python2.7/socket.py", line 303, in flush
    self._sock.sendall(view[write_offset:write_offset+buffer_size])
error: [Errno 32] Broken pipe
----------------------------------------
</code></pre>

<p>Any ideas?</p>
