Clinet's nonce in TLS
Your confusion comes in who the client's nonce is protecting. The client's nonce protects the client from server replays. Similarly, the server's nonce protects the server from client replays. This makes it so that neither has to remember past nonces. As long as the client is implemented correctly and does not reuse a nonce, a server trying to replay a message will fail.