What is encrypted with the key to give the keystream in CTR mode exactly? I'm learning how CTR mode and there is something I have trouble understanding. If I understood correctly, two components are encrypted with the key, a nonce and a counter concatenated, and it looks like for example a5b1fe9ac36701ba0000000000000003 : 8 bytes of nonce and 8 bytes of the counter padded with zeros. However, I also read that reusing the nonce produce the same keystream and must be avoided. Why is that, since the counter will differ each time ?
The counter is increased for each block in a message. However, it is restarted at zero for the next message. That way only the nonce is required to be different. If you can synchronize the counter over multiple messages then yes, just increasing the counter is secure. However, you would loose the ability to encrypt messages in parallel, and you would somehow need to keep track of the counter when finishing encrypting a message.
Note that the 8 byte nonce and 8 byte counter is just one way to create a unique input for the block cipher. Other schemes / sizes are certainly possible. For instance, you could use the ID of an entity in there to separate those for the same key (a bit for a client &amp; one for the server). Generally however, we keep to a nonce and counter in that order. Domain separation is often performed by using different keys for different purposes.
Beware that it is often up to the user to make sure that the counter doesn't overflow into the nonce. Furthermore, if the nonce is randomized then you should take note of the birthday paradox; it might be a good idea to increase the nonce size compared to the message size. E.g. a 32 bit counter would still leave you with $16 \cdot 2^{32} = 64\ \text{GiB}$  message size. That should even be enough for most high bitrate movies, and it would leave you with a full 96 bits for the nonce. The authenticated cipher mode GCM - which uses CTR mode internally - does exactly specify 64 GiB - 32 bytes as maximum message size, so you can guess the nonce and counter size by that alone.