title: TCP receiving window size higher than net.core.rmem_max
body: <p>I am running <code>iperf</code> measurements between two servers, connected through 10Gbit link. I am trying to correlate the maximum window size that I observe with the system configuration parameters.</p>
  
  <p>In particular, I have observed that the maximum window size is 3 MiB. However, I cannot find the corresponding values in the system files.</p>
  
  <p>By running <code>sysctl -a</code> I get the following values:</p>
  
  <pre><code>net.ipv4.tcp_rmem = 4096        87380   6291456
  net.core.rmem_max = 212992
  </code></pre>
  
  <p>The first value tells us that the maximum receiver window size is 6 MiB. However, TCP tends to allocate twice the requested size, so the maximum receiver window size should be 3 MiB, exactly as I have measured it. From <code>man tcp</code>:</p>
  
  <blockquote>
    <p>Note that TCP actually allocates twice the size of the buffer requested in the setsockopt(2) call, and so a succeeding getsockopt(2) call will not return the same size of buffer as requested in the setsockopt(2) call.  TCP uses the extra space for administrative purposes and  internal  kernel  structures,  and  the /proc file values reflect the larger sizes compared to the actual TCP windows.</p>
  </blockquote>
  
  <p>However, the second value, <code>net.core.rmem_max</code>, states that the maximum receiver window size cannot be more than 208 KiB. And this is supposed to be the hard limit, according to <code>man tcp</code>:</p>
  
  <blockquote>
    <p>tcp_rmem 
    max: the maximum size of the receive buffer used by each TCP socket.  This value <strong>does not override</strong> the  global <code>net.core.rmem_max</code>. This is not used to limit the size of the receive buffer declared using SO_RCVBUF on a socket.</p>
  </blockquote>
  
  <p>So, how come and I observe a maximum window size larger than the one specified in <code>net.core.rmem_max</code>? </p>
  
  <p>NB: I have also calculated the Bandwidth-Latency product: <code>window_size = Bandwidth x RTT</code> which is about 3 MiB (10 Gbps @ 2 msec RTT), thus verifying my traffic capture.</p>