chunked/nltk_chunking/_deepspeed/chunk_142.txt

When the communication is done in fp16 or bf16, it is more likely to be lossy because adding multiple numbers in low precision isn't exact.