title: Generic Netlink unicast from kernel to user fails (-111)
body: <p>(Linux 4.4)</p>
  
  <p>I am trying to get a kernel module to send information to a user process over a Generic Netlink.  It seems that the message is not successfully received by the user process - the nlmsg_unicast function returns with -111.</p>
  
  <p>Here is what I know:</p>
  
  <ul>
  <li>The kernel module successfully registers a Generic Netlink family - it prints a message in the syslog indicating the (autogenerated) family ID (which is always 26).</li>
  <li>The user process successfully discovers the family ID (26).</li>
  <li>The user process sends a sort of "I'm alive" command to the kernel module, which successfully logs the (auto-selected) port ID of the user process - I know from messages printed by both the user process and the kernel module that the correct portID is resolved.</li>
  <li>Subsequently, the kernel, upon an event, tries to send a message to the resolved portID over the Generic Netlink family that had been set up.</li>
  <li>The user process never receives the message (it never enters the callback function; in fact, I don't think mnl_socket_recvfrom ever returns).  In the kernel module, the nlmsg_unicast function returns with -111.</li>
  </ul>
  
  <p>I am using libmnl in the user process (as you might have guessed from my allusion to mnl_socket_recvfrom).</p>
  
  <p>uname -a<br> Linux yaron-VirtualBox 4.4.0-57-generic #78-Ubuntu SMP Fri Dec 9 23:50:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux</p>
  
  <p>Here is, essentially, my send code in the kernel:</p>
  
  <pre><code>    struct sk_buff *msg;
      struct sock *socket;
      struct netlink_kernel_cfg nlCfg = {
          .groups = 1,
          .flags = 0,
          .input = NULL,
          .cb_mutex = NULL,
          .bind = NULL,
          .unbind = NULL,
          .compare = NULL,
      };
      void *msg_head;
      int retval;
      struct net init_net;
  
              /* Open a socket */
              socket = netlink_kernel_create(&amp;init_net, NETLINK_GENERIC, &amp;nlCfg);
              if (socket == NULL) goto CmdFail;
  
              /* Allocate space */
              msg = genlmsg_new(NLMSG_GOODSIZE, GFP_KERNEL);
              if (msg == NULL) goto CmdFail;
  
              /* Generate message header
               * arguments of genlmsg_put: 
               *    struct sk_buff *, 
               *    int portID,  &lt;-- this is sender portID
               *    int netlinkSeqNum,
               *    struct genl_family *, 
               *    int flags, 
               *    u8 command_idx         */
              msg_head = genlmsg_put(msg, 0, ++netlinkSeqNum, &amp;genlFamily, 0, MYFAMILY_CMD_MYMSG);
              if (msg_head == NULL) goto CmdFail;
  
              /* Add a MYFAMILY_ATTR_MYCMD attribute (command to be sent) */
              retval = nla_put_string(msg, MYFAMILY_ATTR_MYMSG, "Temporary message");
              if (retval != 0) goto CmdFail;
  
              /* Finalize the message */
              genlmsg_end(msg, msg_head);  /* void inline function - no return value */
  
              /* Send the message */
              retval = nlmsg_unicast(socket, msg, userNetlinkPortID);
              printk("nlmsg_unicast returned %d\n", retval);
              if (retval != 0) goto CmdFail;
  
              netlink_kernel_release(socket);
  
              return;
  
  
  CmdFail:
      printk(KERN_ALERT "*** Failed to send command !\n");
      netlink_kernel_release(socket);
      return;
  </code></pre>
  
  <p>Here is, essentially, my receive code in the user process:</p>
  
  <pre><code>char bufferHdr[getpagesize()];
  struct nlmsghdr *nlHeader;
  struct genlmsghdr *nlHeaderExtraHdr;
  int numBytes, seq, ret_val;
  
  // Set up the header.
  // Function mnl_nlmsg_put_header will zero out a length of bufferHdr sufficient to hold a Netlink header,
  // and initialize the nlmsg_len field in that space to the size of a header.
  // It returns a pointer to bufferHdr.
  if ( (nlHeader = mnl_nlmsg_put_header(bufferHdr)) != (struct nlmsghdr *) bufferHdr ) {
      perror("mnl_nlmsg_put_header failed");
      exit(EXIT_FAILURE);
  }
  nlHeader-&gt;nlmsg_type = genetlinkFamilyID;
  
  // Function mnl_nlmsg_put_extra_header extends the header, to allow for these extra fields.
  if ( (nlHeaderExtraHdr = (struct genlmsghdr *) mnl_nlmsg_put_extra_header(nlHeader, sizeof(struct genlmsghdr))) != (struct genlmsghdr *) (bufferHdr + sizeof(struct nlmsghdr)) ) {
      perror("mnl_nlmsg_put_extra_header failed");
      exit(EXIT_FAILURE);
  }
  // No command to set
  
  // No attributes to set
  
  // Wait for a message, and process it
  while (1) {
      numBytes = mnl_socket_recvfrom(nlSocket, bufferHdr, sizeof(bufferHdr));
      if (numBytes == -1) {
          perror("mnl_socket_recvfrom returned error");
          break;
      }
      // Callback run queue handler - use it to call getMsgCallback
      std::cout &lt;&lt; "received a msg, handling it" &lt;&lt; std::endl;
      ret_val = mnl_cb_run(bufferHdr, numBytes, seq, portid, getMsgCallback, NULL);
      if (ret_val == -1) {
          //perror("mnl_cb_run failed");
          break;
      } else if (ret_val == 0)
          break;
  
  }
  
  return ret_val;
  </code></pre>
  
  <p><hr>
  <strong>ADDENDUM:</strong>
  Having scoured through the kernel source code some more (on elixir.free-electrons.com), I'm guessing that my message never even gets to the user process; suggestions for debugging would be appreciated.</p>
  
  <p>Here is what I'm seeing: <code>nlmsg_unicast</code> calls <code>netlink_unicast</code>, which in turn calls <code>netlink_getsockbyportid</code>, which looks like this:</p>
  
  <pre><code>static struct sock *netlink_getsockbyportid(struct sock *ssk, u32 portid)
  {
      struct sock *sock;
      struct netlink_sock *nlk;
  
      sock = netlink_lookup(sock_net(ssk), ssk-&gt;sk_protocol, portid);
      if (!sock)
          return ERR_PTR(-ECONNREFUSED);
  
      /* Don't bother queuing skb if kernel socket has no input function */
      nlk = nlk_sk(sock);
      if (sock-&gt;sk_state == NETLINK_CONNECTED &amp;&amp;
          nlk-&gt;dst_portid != nlk_sk(ssk)-&gt;portid) {
          sock_put(sock);
          return ERR_PTR(-ECONNREFUSED);
      }
      return sock;
  }
  </code></pre>
  
  <p>I'm guessing one of the two conditions here for punting and returning -ECONNREFUSED is triggered.</p>
  
  <p>Any suggestions for how I can debug whether either of these conditions is true?  It doesn't look like I can call <code>netlink_lookup</code> or <code>nlk_sk</code> directly from my module code - I guess the symbols are not exposed - nor their subfunctions - a whole lot of symbols are buried in af_netlink.h and af_netlink.c, and I guess the symbols are not available when building your external module, at least the normal way.  (It doesn't look like af_netlink.h is available as part of the distro.)</p>