title: Database crashed, need help recovering
body: <p>I try to restart, and get this error:</p>
  
  <blockquote>
    <p>ERROR! MySQL server PID file could not be found!
    Starting MySQL. ERROR! The server quit without updating PID file (/var/lib/mysql/www.mysitecom.pid).</p>
  </blockquote>
  
  <p>Look at the log.. see a ton of this : </p>
  
  <pre><code>140502 14:10:24 InnoDB: The InnoDB memory heap is disabled
  140502 14:10:24 InnoDB: Mutexes and rw_locks use GCC atomic builtins
  140502 14:10:24 InnoDB: Compressed tables use zlib 1.2.3
  140502 14:10:24 InnoDB: Using Linux native AIO
  140502 14:10:24 InnoDB: Initializing buffer pool, size = 128.0M
  140502 14:10:24 InnoDB: Completed initialization of buffer pool
  140502 14:10:24 InnoDB: highest supported file format is Barracuda.
  InnoDB: The log sequence number in ibdata files does not match
  InnoDB: the log sequence number in the ib_logfiles!
  140502 14:10:24  InnoDB: Database was not shut down normally!
  InnoDB: Starting crash recovery.
  InnoDB: Reading tablespace information from the .ibd files...
  InnoDB: Error: trying to add tablespace 12 of name './roundcube/searches.ibd'
  InnoDB: to the tablespace memory cache, but tablespace
  InnoDB: 12 of name './d6nspw4f_dell/wp_users.ibd' already exists in the tablespace
  InnoDB: memory cache!
  </code></pre>
  
  <p>Was able to restart by adding <code>innodb_force_recovery = 4</code> to the sql config file, but all sites that touch the db are now broken and can’t be written to.</p>
  
  <p>When I try to do a mysql dump of all tables, I get the error : </p>
  
  <pre><code>mysqldump: Got error: 1146: Table 'x3hmcpl_hmcpl.blocked_ips' doesn't exist when using LOCK TABLES
  </code></pre>
  
  <p>Any help at all would be appreciated. We don't have a database guy here and I'm stuck with the task of trying to fix it as quickly as possible.</p>