title: How to translate X11 keycode back to scancode or hid usage id reliably
body: <p>Almost self resolved.  </p>
  
  <h1>Background info</h1>
  
  <h2>linux keycode</h2>
  
  <p>The below is from: linux_xmodmap_tutorial</p>
  
  <ul>
  <li><p>Scancode → a number your keyboard sends to a computer. For USB
  keyboards, it's defined by the USB standard. </p></li>
  <li><p>Keycode → a number used by Linux kernel to represent key (or mouse
  button/wheel). The Linux kernel translates scancode to keycode.</p></li>
  <li><p>Keysym → a name for a key. keysym is received by X11 applications.</p></li>
  </ul>
  
  <p>I could be wrong but what <code>sudo evtest</code> prints seems like scancode set 1 and since <code>/dev/input/event*</code> outputs are similar, at least with my system scancodes and linux keycodes are mostly same.</p>
  
  <h2>X11 keycode</h2>
  
  <p>xev command prints keycode and keysym and I observed keycode does not change regardless of keyboard layout I use;qwerty and dvorak layouts produce a different keysym but the same keycode.<br>
  Nevertheless, it's not a linux keycode nor hid usage id nor any one of the scancode sets;set1 nor set2 nor set3.  </p>
  
  <h2>how is linux keycode generated?</h2>
  
  <p>According to arch wiki(I cannot post links):<br>
  <code>/etc/udev/hwdb.bin</code> translates scancode to keycode.</p>
  
  <p>The Obvious snugs with this are:</p>
  
  <ol>
  <li>the wiki says there are non-default *.hwdb files that can be
  compiled to <code>hwdb.bin</code></li>
  <li><code>hwdb.bin</code> can detect different keyboards and may translates
  a scancode to a keycode differently.</li>
  </ol>
  
  <p>For ubuntu, the only hwdb file I can find is: <code>/etc/udev/hwdb.d/61-keyboard-local.hwdb</code> and that is almost empty. So, by default ubuntu preserves the default scancodes and use them as keycodes I believe.   </p>
  
  <h2>How is X11 keycode generated</h2>
  
  <p>Some hacker left us a clue here:<a href="https://www.charvolant.org/doug/xkb/html/index.html" rel="noreferrer">https://www.charvolant.org/doug/xkb/html/index.html</a><br>
  The scancode to keycode mapping rule files(which one is being used is the question) are under: <code>/usr/share/X11/xkb/keycodes</code></p>
  
  <p><code>setxkbmap -print -verbose 10</code> prints</p>
  
  <pre><code>Trying to load rules file ./rules/evdev...
  Trying to load rules file /usr/share/X11/xkb/rules/evdev...
  </code></pre>
  
  <p>so, the above files should be controlling the loading behavior for Xkb(Linux changes a lot, so should not expect the same in the future though). </p>
  
  <p>Example:<br>
  setxkbmap -print -verbose 10</p>
  
  <pre><code>Setting verbose level to 10
  locale is C
  Trying to load rules file ./rules/evdev...
  Trying to load rules file /usr/share/X11/xkb/rules/evdev...
  Success.
  Applied rules from evdev:
  rules:      evdev
  model:      pc105
  layout:     us,us
  variant:    dvp,
  Trying to build keymap using the following components:
  keycodes:   evdev+aliases(qwerty)
  types:      complete
  compat:     complete
  symbols:    pc+us(dvp)+us:2+inet(evdev)
  geometry:   pc(pc105)
  xkb_keymap {
      xkb_keycodes  { include "evdev+aliases(qwerty)" };
      xkb_types     { include "complete"  };
      xkb_compat    { include "complete"  };
      xkb_symbols   { include "pc+us(dvp)+us:2+inet(evdev)"   };
      xkb_geometry  { include "pc(pc105)" };
  </code></pre>
  
  <p>A section in <code>/usr/share/X11/xkb/rules/evdev</code> which is likely controlling the keycode file loading behaviour:</p>
  
  <pre><code>! model         =       keycodes
    pc98          =       evdev(pc98)
    applealu_jis  =       evdev+macintosh(jisevdev)
    olpc          =       evdev+olpc(olpc)
    olpcm         =       evdev+olpc(olpcm)
    *             =       evdev
  
  ! layout[1]     =       keycodes
    $azerty       =       +aliases(azerty)
    $qwertz       =       +aliases(qwertz)
    *             =       +aliases(qwerty)
  </code></pre>
  
  <h3>My guess</h3>
  
  <p>XServer looks at the keyboard's vendor id and device id and applies different rules for translating scancode to keycode.</p>
  
  <h3>Can I get scancode from X11 keycode?</h3>
  
  <p><code>xkb/keycodes/evdev</code> seems to be used as translation rule for most keyboards, using it might does the trick.</p>
  
  <h3>Is it reliable?</h3>
  
  <p>No<br>
  The traditional X11 the core protocol does not provide the vendor id nor device id of the keyboard that generated a keyboard event. </p>
  
  <h2>Potential solution?</h2>
  
  <p>XI2 - The X Input Extension 2.x<br>
  <a href="https://www.x.org/releases/X11R7.7/doc/inputproto/XI2proto.txt" rel="noreferrer">https://www.x.org/releases/X11R7.7/doc/inputproto/XI2proto.txt</a></p>
  
  <pre><code>A RawEvent provides the information provided by the driver to the
  client. RawEvent provides both the raw data as supplied by the driver and
  transformed data as used in the server.   
  </code></pre>
  
  <h3>Now I have tried RawKeyPress</h3>
  
  <p>I could not extract hid usage id nor scancode;though maybe I just overlooked them.<br>
  <a href="https://stackoverflow.com/questions/45809046/capture-xi2-rawkeypress-event-and-interpreting-it-with-python/45847216#45847216">Capture XI2 RawKeyPress event and interpreting it with python</a></p>