title: Ensuring all the data gets received through UDP socket in Python
body: <p>Possibly related questions that seem close but don't describe my issue as I understand it:</p>
  
  <ul>
  <li><a href="https://stackoverflow.com/questions/11026316/reading-all-the-data-from-a-udp-socket">Reading all the data from a UDP socket</a></li>
  <li><a href="https://stackoverflow.com/questions/11536613/python-udp-socket-semi-randomly-failing-to-receive">Python UDP socket semi-randomly failing to receive</a></li>
  <li><a href="https://stackoverflow.com/questions/9969259/python-raw-socket-listening-for-udp-packets-only-half-of-the-packets-received">Python raw socket listening for UDP packets; only half of the packets received</a></li>
  </ul>
  
  <h3>problem</h3>
  
  <p>Long file sent line by line doesn't go all the way through UDP over loopback.</p>
  
  <h3>long story</h3>
  
  <p>I have a long file consisting of lines and breaks that is identical to what I will get from another program over UDP locally. Let me emphasize that <strong>the program sending the packets will do so over UDP (there is no choice here), and cannot be feasibly modified to process ACK requests etc. while sending.</strong></p>
  
  <p>It looks like this (this is the tail):</p>
  
  <pre><code>StimulusTime 56398
  
  Signal(0,2) -79.5457
  
  Signal(0,4) -81.7426
  
  Signal(0,6) -83.9978
  
  Signal(0,9) -63.3755
  
  Signal(0,11) -15.6045
  
  Signal(0,13) 31.1299
  
  Signal(0,16) 75.7539
  
  Signal(0,18) 98.301
  
  Signal(0,20) 98.301
  
  Signal(0,22) 48.4546
  
  Signal(0,25) 3.73159
  
  Signal(0,27) -49.9798
  
  Signal(0,29) -77.8449
  
  Signal(1,0) -22.0332
  
  Signal(1,2) -60.6384
  
  Signal(1,4) -98.0858
  
  Signal(1,6) -86.4579
  
  Signal(1,9) -68.9173
  
  Signal(1,11) -31.5552
  
  Signal(1,13) 35.2906
  
  Signal(1,16) 77.0686
  
  Signal(1,18) 96.3836
  
  Signal(1,20) 95.7246
  
  Signal(1,23) 25.6074
  
  Signal(1,25) -20.2101
  
  Signal(1,27) -60.2933
  
  Signal(1,29) -83.8169
  
  Signal(2,0) -31.8826
  
  Signal(2,2) -53.5045
  
  Signal(2,4) -89.9895
  
  Signal(2,7) -84.4503
  
  Signal(2,9) -59.7016
  
  Signal(2,11) -12.8569
  
  Signal(2,13) 28.857
  
  Signal(2,15) 58.0577
  
  Signal(2,18) 96.4222
  
  Signal(2,20) 79.783
  
  Signal(2,22) 58.6463
  
  Signal(2,25) -3.24883
  
  Signal(2,27) -45.5
  
  Signal(2,29) -88.8937
  
  Signal(3,0) -18.6625
  
  Signal(3,2) -53.3978
  
  Signal(1,16) 58.784
  
  Signal(1,17) 44.7782
  
  Signal(1,18) 6.247
  
  Signal(1,19) -12.0855
  
  Signal(1,20) -33.7644
  
  Signal(1,21) -49.4406
  
  Signal(1,22) -67.5791
  
  Signal(1,23) -92.0336
  
  Signal(1,24) -93.9841
  
  END
  </code></pre>
  
  <p>I wrote code that takes this file and sends it a line at a time over UDP locally, and then code that receives it and parses it based on the data type.</p>
  
  <p>Sender:</p>
  
  <pre><code>import socket
  import sys
  
  # Sends udp test data piped in from STDIN to the listener.
  # ex: cat sampleoutput.txt | python testsender.py 
  
  UDP_IP = "127.0.0.1"
  UDP_PORT = 5000
  
  print "UDP target IP:", UDP_IP
  print "UDP target port:", UDP_PORT
  
  sock = socket.socket(socket.AF_INET, # Internet
                       socket.SOCK_DGRAM) # UDP
  
  # Send from stdin
  if len(sys.argv) &lt; 2:
    while True:
      line = sys.stdin.readline()
      if line:
        sock.sendto(line, (UDP_IP, UDP_PORT))
      else:
        break
  
    # get from file arg
  else:
    myfile = open(str(sys.argv[1]), "r")
    while True:
      line = myfile.readline()
      if line:
        sock.sendto(line, (UDP_IP, UDP_PORT))
      else:
        break
  
  
  sock.close()
  </code></pre>
  
  <p>Listener:</p>
  
  <pre><code>import socket
  from array import array
  
  UDP_IP = "127.0.0.1"
  UDP_PORT = 5000
  
  sock = socket.socket(socket.AF_INET,  # Internet
                       socket.SOCK_DGRAM)  # UDP
  sock.bind((UDP_IP, UDP_PORT))
  
  while True:
      data, addr = sock.recvfrom(1024)  # buffer size arg
  
      print data
      # write to file for later testing
      # file = open("testdummy.txt", "a")
      # file.write(data)
      if data == "END\n":
          break
  </code></pre>
  
  <p>I was able to use the above listener to produce the test file from the original program, so it should work. Unfortunately, it fails around 500 lines of payload, as tested by <code>tail -n 500 testdummy.txt | python testsender.py</code>, although it's somewhat random. Specifically, the listener does not receive all of the sent lines before the sender exits, leaving it hanging, waiting for the <code>"END\n"</code> string.</p>
  
  <p>As I understand it, the socket is already in blocking mode--how can I prevent this from occurring?</p>