title: TCP_MAXSEG inaccurate? (Was: Linux path MTU probing not working on accept():ed socket if requested using setsockopt())
body: <p>Question update: In addition to the problem below, it seems our client/server application using the Linux PLPMTUD mechanism gets too large path MTU. Has anyone seen this, i.e. actual path MTU being 1500, but <code>getsockopt()</code> w <code>TCP_MAXSEG</code> returning the MTU:s of the endpoints, in our case 3000? I have tried turning of GRO, GSO and TSO with ethtool but the error persists. Normal ping only manages to push through packets 1472 bytes or smaller. Also worth mentioning is that PLPMTUD works perfectly for smaller MTU:s. For example, w endpoints at 1500 MTU and one interface of the intermediate router set to e.g 1200 MTU, the kernel TCP probes and reports correct <code>TCP_MAXSEG</code> (1200 - headers).</p>
  <p>I am using the Linux RFC4821-compliant packetization layer path MTU discovery in an application. Basically, the client does a setsockopt on a TCP socket:</p>
  <pre><code>setsockopt(fd, SOL_IP, IP_MTU_DISCOVER, &amp;sopt, sizeof(sopt));
  </code></pre>
  <p>with option value set to <code>IP_PMTUDISC_PROBE</code>. The <code>setsockopt()</code> does not return an error.</p>
  <p>The client sends large tcp packets to a discard server, and the path MTU is calibrated by Linux kernel - tcpdump shows tcp packets with DF bit set being sent, packet size varies until the kernel knows the path MTU. However, to get this to work in the other direction (listening server accept:ing connections from clients, sending data and calibrating PMTU in direction from server to client) I have to set the global option for tcp path mtu discovery, <code>/proc/sys/net/ipv4/tcp_mtu_probing</code>. If I do not, server will stupidly continue to send too large packets, which get discarded by an intermediate router without ICMP sent back. Both endpoints have an MTU set to 3000, while the intermediate hops have MTU 1500.</p>
  <p>I hope someone has an idea on what goes wrong. If more info is needed, let me know and I edit the question. Problems exist on both Linux kernel 4.2.0 and 3.19.0, both are stock Kubuntu LTS kernels. (x86/x86-64)</p>
  <p>I do set the same socket option server-side as well, on all accept:ed sockets, before sending data in reverse direction.</p>