Regarding ICMP "Fragmentation needed, DF bit set" or ICMP packet too big message
Question: I'm injecting ICMP "Fragmentation needed, DF bit set" into the server and
ideally server should start sending packets with the size mentioned in the
field 'next-hop MTU' in ICMP. But this is not working.

Here is the server code:

    
    
    #!/usr/bin/env python 
    import socket               # Import socket module
    import time
    import os
    
    range= [1,2,3,4,5,6,7,8,9]
    s = socket.socket()         # Create a socket object
    host = '192.168.0.17'                   # Get local machine name
    port = 12349               # Reserve a port for your service.
    s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
    s.bind((host, port))        # Bind to the port
    rand_string = os.urandom(1600)
    
    s.listen(5)                 # Now wait for client connection.
    while True:
       c, addr = s.accept()     # Establish connection with client.
       print 'Got connection from', addr
       for i in range:
        c.sendall(rand_string)
            time.sleep(5)
       c.close()
    

Here is the client code:

    
    
    #!/usr/bin/python           # This is client.py file
    
    import socket               # Import socket module
    
    s = socket.socket()         # Create a socket object
    host = '192.168.0.17' # Get local machine name
    port = 12348              # Reserve a port for your service.
    s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
    s.connect((host, port))
    while 1:
        print s.recv(1024)
    s.close()
    

Scapy to inject ICMP:

    
    
    ###[ IP ]###
      version= 4
      ihl= None
      tos= 0x0
      len= None
      id= 1
      flags= DF
      frag= 0
      ttl= 64
      proto= ip
      chksum= None
      src= 192.168.0.45
      dst= 192.168.0.17
      \options\
    ###[ ICMP ]###
      type= dest-unreach
      code= fragmentation-needed
      chksum= None
      unused= 1300
    
    Send(ip/icmp)
    

Unused field shows as next-hop MTU in wireshark. Is server smart enough to
check that DF Bit was not set when it was communicating with client and it is
still receiving ICMP "Fragmentation needed, DF bit set" message? If it is not
then why is server not reducing its packet size from 1500 to 1300?


Answer: First of all, let's answer your first question (is ICMP sent over TCP?).

ICMP runs directly over IP, as specified in [RFC
792](http://www.ietf.org/rfc/rfc0792.txt):

> ICMP messages are sent using the basic IP header.

This can be a bit confusing as [ICMP is classified as a network layer protocol
rather than a transport layer
protocol](http://stackoverflow.com/a/26275365/3903832) but it makes sense when
taking into account that it's merely an addition to IP to carry error, routing
and control messages and data. Thus, it can't rely on the TCP layer to
transfer itself since the TCP layer depends on the IP layer which ICMP helps
to manage and troubleshoot.

* * *

Now, let's deal with your second question (How does TCP come to know about the
MTU if ICMP isn't sent over TCP?). I've tried to answer this question to the
best of my understanding, with reliance on official specifications, but
perhaps the best approach would be to analyze some open source network stack
implementation in order to see what's really going on...

The TCP layer may come to know of the path's MTU value even though the ICMP
message is not layered upon TCP. It's up to the implementation of OS the
network stack to notify the TCP layer of the MTU so it can then use this value
to update its MSS value.

[RFC 1122](http://tools.ietf.org/html/rfc1122) requires that the ICMP message
includes the IP header as well as the first 8 bytes of the problematic
datagram that triggered that ICMP message:

> Every ICMP error message includes the Internet header and at least the first
> 8 data octets of the datagram that triggered the error; more than 8 octets
> MAY be sent; this header and data MUST be unchanged from the received
> datagram.
>
> In those cases where the Internet layer is required to pass an ICMP error
> message to the transport layer, the IP protocol number MUST be extracted
> from the original header and used to select the appropriate transport
> protocol entity to handle the error.

This illustrates how the OS can pinpoint the TCP connection whose MSS should
be updated, as these 8 bytes include the source and destination ports.

RFC 1122 also states that there MUST be a mechanism by which the transport
layer can learn the maximum transport-layer message size that may be sent for
a given {source, destination, TOS} triplet. Therefore, I assume that once an
`ICMP Fragmentation needed and DF set` error message is received, the MTU
value is somehow made available to the TCP layer that can use it to update its
MSS value.

Furthermore, I think that the application layer that instantiated the TCP
connection and taking use of it may handle such messages as well and fragment
the packets at a higher level. The application may open a socket that expects
ICMP messages and act accordingly when such are received. However, fragmenting
packets at the application layer is totally transparent to the TCP & IP
layers. Note that most applications would allow the TCP & IP layers to handle
this situation by themselves.

However, once an `ICMP Fragmentation needed and DF set` error message is
received by a host, its behavior as dictated by the lower layers is not
conclusive.

[RFC 5927, section 2.2](http://tools.ietf.org/html/rfc5927#section-2.2) refers
to RFC 1122, section 4.2.3.9 which states that TCP should abort the connection
when an `ICMP Fragmentation needed and DF set` error message is passed up from
the IP layer, since it signifies a hard error condition. The RFC states that
the host should implement this behavior, but it is not a must (section 4.2.5).
This RFC also states in section 3.2.2.1 that a Destination Unreachable message
that is received MUST be reported to the TCP layer. Implementing both of these
would result in the destruction of a TCP connection when an `ICMP
Fragmentation needed and DF set` error message is received on that connection,
which doesn't make any sense, and is clearly not the desired behavior.

On the other hand, [RFC 1191](http://www.ietf.org/rfc/rfc1191.txt) states
[this](http://seclists.org/bugtraq/2001/Jan/219) in regard to the required
behavior:

> RFC 1191 does not outline a specific behavior that is expected from the
> sending host, because different applications may have different
> requirements, and different implementation architectures may favor different
> strategies [This leaves a room for this method-OA].
>
> The only required behavior is that a host must attempt to avoid sending more
> messages with the same PMTU value in the near future. A host can either
> cease setting the Don't Fragment bit in the IP header (and allow
> fragmentation by the routers in the way) or reduce the datagram size. The
> better strategy would be to lower the message size because fragmentation
> will cause more traffic and consume more Internet resources.

For conclusion, I think that the specification is not definitive in regard to
the required behavior from a host upon receipt of an `ICMP Fragmentation
needed and DF set` error message. My guess is that both layers (IP & TCP) are
notified of the message in order to update their MTU & MSS values,
respectively and that one of them takes upon the responsibility of
retransmitting the problematic packet in smaller chunks.

* * *

Lastly, regarding your implementation, I think that for full compliance with
RFC 1122, you should update the ICMP message to include the IP header of the
problematic packet, as well as its next 8 bytes (though you may include more
than just the first 8 bytes). Moreover, you should verify that the ICMP
message is received before the corresponding ACK for the packet to which that
ICMP message refers. In fact, just in order to be on the safe side, I would
abolish that ACK altogether.

[Here](http://www.packetlevel.ch/html/scapy/download/pmtu.py) is a sample
implementation of how the ICMP message should be built. If sending the ICMP
message as a response to one of the TCP packets fails, I suggest you try
sending the ICMP message before even receiving the TCP packet to which it
relates at first, in order to assure it is received before the ACK. Only if
that fails as well, try abolishing the ACK altogether.

