Libraries module. Where to extract? Where should I be extracting the Libraries module, sites/all/modules or sites/all/libraries?
As all the modules, it goes in site/all/modules, or in $config/modules, where $config is the directory containing the settings.php file currently used for the site. (See drupal_system_listing() to understand where Drupal looks for modules.)

sites/all/libraries is the directory where the third-party library files should go. (Although, that is not the only directory where the Libraries module looks for them.)