1 Process A open a file F for write.
2 Somebody else called recoverLease against F.
The root cause of the missing block is that recoverLease increased gen count of blocks on datanode whereas the gen count on Namenode is reset by close in step 3. I think we should check if the last block is under recovery when trying to close a file. If so we should just throw an exception to client quickly.
Although the issue is found on a 2.4 HDFS, it looks like the issue also exist on the trunk from code. I'll post a patch soon.
HDFS-3584 Blocks are getting marked as corrupt with append operation under high load.