title: Linux kernel module: How to reinject packets the kernel considers as NF_STOLEN?
body: <p>Good evening. Posting to this site is new for me, but I've been a thankful reader, who has learned a lot from this forum, for quite a while now. This is the first time I am encountering a problem I neither can solve myself, nor with the help of entries which already exist on Stackoverflow or any other resource the internet offers.</p>
  
  <p>I hope you mind to help me again (and that, from now on, I will be able to help others as well, because I feel like I grew to a point where I can start being a writing member here).</p>
  
  <h1>The issue:</h1>
  
  <p>I am working on a kernel module. It's purpose is to steal incoming packets with a certain source IP from the kernel with a PRE_ROUTING netfilter hook. Only TCP packets are interessting for it.</p>
  
  <p>Right now, the hook ony reinjects the packets to the normal kernel packet processing routines via dev_queue_xmit() and returns NF_STOLEN for the packet to the kernel. Packets from other source addresses are not reinjected, but ignored by returning NF_ACCEPT for them instead of NF_STOLEN.</p>
  
  <p>The kernel module as well stores the TCP seq number of every stolen packet in order to determine whether an incoming packet from the mentioned IP is new, or was already modified and reinjected via dev_queue_xmit(), as these packets traverse the hook again.</p>
  
  <h1>What is working yet:</h1>
  
  <ol>
  <li>Module gets loaded</li>
  <li>Hook is registered</li>
  <li>Hook is called for every packet.</li>
  <li>Hook can determine if the packets SRC IP is the IP I am looking for.</li>
  <li>Hook returns NF_ACCEPT for packets with other source address</li>
  <li>Packets with the source address are reinjected while NF_STOLEN is returned for them</li>
  <li>Reinjected packets traverse the hook again and are ignored </li>
  </ol>
  
  <h1>The problem</h1>
  
  <p>When I visit the IP with my browser after loading the module, my IP stack seems to be crashing. I can not ping any address anymore. The module logs that it encountered packets from the relevant IP, and that it requeued them AND that it found an already known packet afterwards (so everything seems fine), but still: no proper connection to the site/any other address.</p>
  
  <h1>Here is the hook code:</h1>
  
  <pre><code>static unsigned int hook(unsigned int hooknum, struct sk_buff *skb, const struct net_device *in, const struct net_device *out, int (*okfn)(struct sk_buff *))
  {
      struct iphdr *iph;
      struct tcphdr *tcph;
      unsigned int i;
  
      if(!skb)
          return NF_ACCEPT;
  
      iph = (struct iphdr *)skb_network_header(skb);
      if(!iph || !(iph-&gt;saddr) || iph-&gt;saddr != *(unsigned int*)suspicious_ip)
          return NF_ACCEPT;
  
      tcph = (struct tcphdr *)skb_transport_header(skb);
      for(i=0; i &lt; number_of_known_packets; i++)
      {
          if(tcph-&gt;seq == *(already_known_packets+i))
          {
              debug("Already known packet");
              return NF_ACCEPT;
          }
      }
      debug("New packet");
      printk("seq: %u\n", tcph-&gt;seq);
  
      if((number_of_known_packets + 1) * 4 &gt;= memory_allocated_for_known_packets) 
          imba_realloc(500*4);
  
      *(already_known_packets+number_of_known_packets++) = tcph-&gt;seq; 
      debug("Requeuing packet");
  
      // once the requeuing is working proper, I want to manipulate the payload as well
      printk("Result: %i", dev_queue_xmit(skb));
      return NF_STOLEN;
  
  
  }
  </code></pre>
  
  <h1>How the hook is registered:</h1>
  
  <pre><code>static struct nf_hook_ops nfho;
  
  int init_module(void)
  {
      debug("module loaded");
  
      already_known_packets = kmalloc(memory_allocated_for_known_packets, GFP_KERNEL);
      debug("initial memory allocated");
  
      nfho.hook = hook;
      nfho.hooknum = NF_INET_PRE_ROUTING;
      nfho.pf = PF_INET;
      nfho.priority = 1;
  
      nf_register_hook(&amp;nfho);
  
      debug("hook registered");
  
      return 0;
  }
  </code></pre>
  
  <h1>syslog:</h1>
  
  <pre><code>Sep 21 13:11:43 linux kernel: [ 3298.937902] [PACKET PROXY] module loaded
  Sep 21 13:11:43 linux kernel: [ 3298.937907] [PACKET PROXY] initial memory allocated
  Sep 21 13:11:43 linux kernel: [ 3298.937931] [PACKET PROXY] hook registered
  Sep 21 13:11:49 linux kernel: [ 3305.415404] [PACKET PROXY] New packet
  Sep 21 13:11:49 linux kernel: [ 3305.415410] seq: 1538346824
  Sep 21 13:11:49 linux kernel: [ 3305.415412] [PACKET PROXY] Requeuing packet
  Sep 21 13:11:49 linux kernel: [ 3305.415430] Result: 0
  Sep 21 13:11:49 linux kernel: [ 3305.415440] [PACKET PROXY] New packet
  Sep 21 13:11:49 linux kernel: [ 3305.415441] seq: 618234741
  Sep 21 13:11:49 linux kernel: [ 3305.415442] [PACKET PROXY] Requeuing packet
  Sep 21 13:11:49 linux kernel: [ 3305.415447] Result: 0
  Sep 21 13:11:49 linux kernel: [ 3305.421440] [PACKET PROXY] New packet
  Sep 21 13:11:49 linux kernel: [ 3305.421452] seq: 2129598066
  Sep 21 13:11:49 linux kernel: [ 3305.421458] [PACKET PROXY] Requeuing packet
  Sep 21 13:11:49 linux kernel: [ 3305.421477] Result: 0
  Sep 21 13:11:49 linux kernel: [ 3305.427449] [PACKET PROXY] New packet
  Sep 21 13:11:49 linux kernel: [ 3305.427456] seq: 2327127721
  Sep 21 13:11:49 linux kernel: [ 3305.427458] [PACKET PROXY] Requeuing packet
  Sep 21 13:11:49 linux kernel: [ 3305.427466] Result: 0
  Sep 21 13:11:49 linux kernel: [ 3305.427470] [PACKET PROXY] New packet
  Sep 21 13:11:49 linux kernel: [ 3305.427471] seq: 1333567182
  Sep 21 13:11:49 linux kernel: [ 3305.427473] [PACKET PROXY] Requeuing packet
  Sep 21 13:11:49 linux kernel: [ 3305.427476] Result: 0
  Sep 21 13:11:49 linux kernel: [ 3305.427494] [PACKET PROXY] New packet
  Sep 21 13:11:49 linux kernel: [ 3305.427502] seq: 2650236943
  Sep 21 13:11:49 linux kernel: [ 3305.427506] [PACKET PROXY] Requeuing packet
  Sep 21 13:11:49 linux kernel: [ 3305.427514] Result: 0
  Sep 21 13:11:49 linux kernel: [ 3305.427522] [PACKET PROXY] New packet
  Sep 21 13:11:49 linux kernel: [ 3305.427533] seq: 444387468
  Sep 21 13:11:49 linux kernel: [ 3305.427534] [PACKET PROXY] Requeuing packet
  Sep 21 13:11:49 linux kernel: [ 3305.427539] Result: 0
  Sep 21 13:11:49 linux kernel: [ 3305.427544] [PACKET PROXY] New packet
  Sep 21 13:11:49 linux kernel: [ 3305.427545] seq: 1405773113
  Sep 21 13:11:49 linux kernel: [ 3305.427547] [PACKET PROXY] Requeuing packet
  Sep 21 13:11:49 linux kernel: [ 3305.427550] Result: 0
  Sep 21 13:11:50 linux kernel: [ 3306.413448] [PACKET PROXY] Already known PACKET
  Sep 21 13:11:50 linux kernel: [ 3306.413641] [PACKET PROXY] Already known PACKET
  Sep 21 13:11:50 linux kernel: [ 3306.414153] [PACKET PROXY] Already known PACKET
  Sep 21 13:11:50 linux kernel: [ 3306.414989] [PACKET PROXY] Already known PACKET
  Sep 21 13:11:50 linux kernel: [ 3306.415102] [PACKET PROXY] Already known PACKET
  Sep 21 13:11:50 linux kernel: [ 3306.417880] [PACKET PROXY] Already known PACKET
  Sep 21 13:11:50 linux kernel: [ 3306.418065] [PACKET PROXY] Already known PACKET
  Sep 21 13:11:50 linux kernel: [ 3306.418134] [PACKET PROXY] Already known PACKET
  Sep 21 13:11:50 linux kernel: [ 3306.433788] [PACKET PROXY] New packet
  Sep 21 13:11:50 linux kernel: [ 3306.433812] seq: 2146375282
  Sep 21 13:11:50 linux kernel: [ 3306.433816] [PACKET PROXY] Requeuing packet
  Sep 21 13:11:50 linux kernel: [ 3306.433850] Result: 0
  Sep 21 13:11:51 linux kernel: [ 3306.441424] [PACKET PROXY] Already known PACKET
  Sep 21 13:11:51 linux kernel: [ 3306.441587] [PACKET PROXY] New packet
  Sep 21 13:11:51 linux kernel: [ 3306.441596] seq: 3958642290
  Sep 21 13:11:51 linux kernel: [ 3306.441610] [PACKET PROXY] Requeuing packet
  Sep 21 13:11:51 linux kernel: [ 3306.441634] Result: 0
  Sep 21 13:11:51 linux kernel: [ 3306.441646] [PACKET PROXY] New packet
  Sep 21 13:11:51 linux kernel: [ 3306.441648] seq: 1476007538
  Sep 21 13:11:51 linux kernel: [ 3306.441652] [PACKET PROXY] Requeuing packet
  Sep 21 13:11:51 linux kernel: [ 3306.441660] Result: 0
  Sep 21 13:11:51 linux kernel: [ 3306.443131] [PACKET PROXY] New packet
  Sep 21 13:11:51 linux kernel: [ 3306.443139] seq: 3288274546
  Sep 21 13:11:51 linux kernel: [ 3306.443148] [PACKET PROXY] Requeuing packet
  Sep 21 13:11:51 linux kernel: [ 3306.443194] Result: 0
  Sep 21 13:11:51 linux kernel: [ 3306.443226] [PACKET PROXY] New packet
  Sep 21 13:11:51 linux kernel: [ 3306.443231] seq: 788862834
  Sep 21 13:11:51 linux kernel: [ 3306.443241] [PACKET PROXY] Requeuing packet
  Sep 21 13:11:51 linux kernel: [ 3306.443258] Result: 0
  Sep 21 13:11:51 linux kernel: [ 3306.443276] [PACKET PROXY] New packet
  Sep 21 13:11:51 linux kernel: [ 3306.443278] seq: 2601129842
  Sep 21 13:11:51 linux kernel: [ 3306.443281] [PACKET PROXY] Requeuing packet
  Sep 21 13:11:51 linux kernel: [ 3306.443286] Result: 0
  Sep 21 13:11:51 linux kernel: [ 3306.443294] [PACKET PROXY] New packet
  Sep 21 13:11:51 linux kernel: [ 3306.443295] seq: 2131695474
  Sep 21 13:11:51 linux kernel: [ 3306.443299] [PACKET PROXY] Requeuing packet
  Sep 21 13:11:51 linux kernel: [ 3306.443305] Result: 0
  Sep 21 13:11:51 linux kernel: [ 3306.443313] [PACKET PROXY] New packet
  Sep 21 13:11:51 linux kernel: [ 3306.443314] seq: 3943962482
  Sep 21 13:11:51 linux kernel: [ 3306.443317] [PACKET PROXY] Requeuing packet
  Sep 21 13:11:51 linux kernel: [ 3306.443320] Result: 0
  Sep 21 13:11:57 linux kernel: [ 3312.685399] [PACKET PROXY] New packet
  Sep 21 13:11:57 linux kernel: [ 3312.685425] seq: 2667014159
  Sep 21 13:11:57 linux kernel: [ 3312.685430] [PACKET PROXY] Requeuing packet
  Sep 21 13:11:57 linux kernel: [ 3312.685463] Result: 0
  </code></pre>