Script on pc only gets 640 characters via serial I experience that if the Arduino (Teensy 3.2) writes a rather long stream of characters to serial, the python script on the other end only receives the first 640 characters. I have found this result both by printing chars, printing lines and by using ArduinoJson's prettyPrintTo(Serial). 

Note, that it does not matter what chunks of characters I read in the Python scripts, whether it is read(5) or read(1000).  

With the Arduino Serial Monitor, I get all the data, ie more than the 640 characters. 

From what I can understand, it is not possible for a sketch to write "too fast" or "too much" to the serial, so there must be something on the pc side receiving the data. However, the Python script for some reason does not get all the data. 

What is the problem here? Where should I look for the problem/solution? Perhaps this is more a Linux or Python question, but I will try here. 

Sketch - printLines()

void printLines() {
  for (int i=0; i&lt;100; i++) {
    // length below is 40 incl end of line character
    Serial.printf("abcdefthijklmnopqrstuvxyz123456789--%03d\n", i);
  }
}

void setup() {
  Serial.begin(9600);
  delay(500);
  printLines();
  //printChars();
  //printJson();
}

void loop() {}


Sketch - printChars()

void printLines() {
  for (int i=0; i&lt;1000; i++) {
    Serial.write('x');
  }
}


Sketch - printJson()

void printJson() {
  StaticJsonBuffer&lt;10000&gt; jsonBuffer;
  JsonArray &amp;root = jsonBuffer.createArray();
  for (int i=0; i&lt;700; i++) {
    root.add(i);
  }
  root.prettyPrintTo(Serial);
}


Python script

import serial

ser = serial.Serial('/dev/ttyACM0')
ser.timeout = 0.5

# PARAMETERS:
# (As long as num_iterations*num_chars &gt;= 640, EXACTLY 640 characters are read)
num_chars = 100
num_iterations = 10

chars_read = 0
for i in range(num_iterations):
    s = ser.read(num_chars)
    chars_read += len(s)
    print("### i: %d, reading %d chars, char %d to %d, len(s): %d, total chars: %d" 
        % (i,num_chars, i * num_chars, i * num_chars + num_chars -1, len(s), chars_read ))


Terminal output

jan@jan:~/Desktop/batlogger/localserver$ sudo python localserver.py 
### i: 0, reading 100 chars, char 0 to 99, len(s): 100, total chars: 100
### i: 1, reading 100 chars, char 100 to 199, len(s): 100, total chars: 200
### i: 2, reading 100 chars, char 200 to 299, len(s): 100, total chars: 300
### i: 3, reading 100 chars, char 300 to 399, len(s): 100, total chars: 400
### i: 4, reading 100 chars, char 400 to 499, len(s): 100, total chars: 500
### i: 5, reading 100 chars, char 500 to 599, len(s): 100, total chars: 600
### i: 6, reading 100 chars, char 600 to 699, len(s): 40, total chars: 640
### i: 7, reading 100 chars, char 700 to 799, len(s): 0, total chars: 640
### i: 8, reading 100 chars, char 800 to 899, len(s): 0, total chars: 640
### i: 9, reading 100 chars, char 900 to 999, len(s): 0, total chars: 640
You are not using a serial connection, you are using a USB CDC/ACM connection, and that is a very very different thing.

With a serial connection when you send data it gets sent regardless of whether there is anything listening for it - the TX IO pin toggles high and low regardless.

However, with a CDC/ACM connection it's only possible to send data when there is something actively listening at the remote end - that is, when the port /dev/ttyACM0 has been opened by an application.

Until that port has been opened any data sent will remain in internal buffers.  These buffers are 64 bytes each in size (that's the size of a CDC/ACM packet), and there are (I would guess) 10 of them on the Teensy3.  After these buffers are full, because there is nothing receiving the data, there can never be any more room made for more data, so instead of blocking like a serial connection does it just discards the data.

So you have filled the buffers with 640 bytes of data and thrown away the rest.  That data just sits there.

Then eventually you open the /dev/ttyACM0 port and the Teensy3 is now able to send CDC/ACM packets through the USB.  And it has 10 of those packets sat there waiting to go out, so it sends them.  So you receive the 640 bytes of data that you had buffered up.

Your mistake is basically not waiting for the port to be opened before sending data to the remote end.  You can test the state of the port (kind of, but that's a whole other subject) by examining the Serial object:

if (Serial) {
    // ... the port is open
}


In general you can usually just wait at the start of your program before starting anything else with a simple while:

while (!Serial); // Wait for the port to be opened