What is USB overhead and why does it reduce the theoretical transfer rate so much?
USB 2.0 is quite inefficient, compared to USB 3.0.

There are Link commands and other transactions, meant to keep the link healthy and operational, that do not carry data. 

There are ACK packets for nearly every received data packet, and because the bus is uni-directional, those disrupt the data transfer.

There is error detection and correction information in the headers, 8/10 bit encoding and other Physical layer header fields, that do not carry payload.

The bus uses broadcast architecture for the downstream channel. That is, every device receives all packets from the host, and is supposed to process only those, directed to it.

Furthermore, the USB2.0 bus is polled, that is the host polls all devices cyclically for new data, them being unable to interrupt the host. This is solved in 3.0 by NRDY and ERDY packet types.

Good thing is, the specification is free and available at usb.org, unlike the C spec, for example. Chapter 3 of the USB 3.0 specification provides quite a nice answer to "How is 3.0 better than 2.0? and What is happening under the hood, at the physical layer?".