title: How to parse / tokenize an SQL statement in Node.js
body: <p>I'm looking for a way to parse / tokenize SQL statement within a Node.js application, in order to:</p>
  
  <ul>
  <li>Tokenize all the "basics" SQL keywords defined in the <a href="https://github.com/marijnh/CodeMirror/tree/master/mode/sql" rel="noreferrer">ISO/IEC 9075</a> standard or <a href="http://codemirror.net/" rel="noreferrer">here</a>.</li>
  <li>Validate the SQL syntax.</li>
  <li>Find out what the query is gonna do (e.g. read or write?).</li>
  </ul>
  
  <p>Do you have any solution or advises peeps?</p>
  
  <p>Linked: <a href="https://stackoverflow.com/questions/13169783/any-javascript-jquery-library-to-validate-sql-statment">Any Javascript/Jquery Library To validate SQL statment?</a>
  <br><br></p>
  
  <hr>
  
  <p>I've done research and I found out some ways to do it:</p>
  
  <p><strong>Using existing node.js libraries</strong></p>
  
  <p>I did a <a href="http://fr.wikipedia.org/wiki/Microsoft_SQL_Server" rel="noreferrer">Google search</a> and I didn't found a consensual and popular library to use. I found those ones: </p>
  
  <ul>
  <li><a href="http://www.mysql.fr/" rel="noreferrer">simple-sql-parser</a> (22 stars on <a href="http://fr.wikipedia.org/wiki/PL/SQL" rel="noreferrer">github</a>, 16 daily download on <a href="http://cassandra.apache.org/" rel="noreferrer">npm</a>)
  
  <ul>
  <li>Supports only SELECT, INSERT, UPDATE and DELETE</li>
  <li>There is a <a href="https://mariadb.org/" rel="noreferrer">v2 branch</a> on the road</li>
  </ul></li>
  <li><a href="https://github.com/forward/sql-parser" rel="noreferrer">sql-parser</a> (90 stars on <a href="https://github.com/forward/sql-parser" rel="noreferrer">github</a>, 6 daily downloads on <a href="https://www.npmjs.org/package/sql-parser" rel="noreferrer">npm</a>)
  
  <ul>
  <li>Only supports basic SELECT statements</li>
  <li>Based on <a href="http://jison.org" rel="noreferrer">jison</a></li>
  </ul></li>
  <li><a href="https://github.com/langpavel/node-sqljs" rel="noreferrer">sqljs</a> (17 stars on <a href="https://github.com/langpavel/node-sqljs" rel="noreferrer">github</a>, 5 daily downloads on <a href="https://www.npmjs.org/package/sqljs" rel="noreferrer">npm</a>)
  
  <ul>
  <li>v0.0.0-3, under development... No documentation at all :)</li>
  </ul></li>
  </ul>
  
  <p>Unfortunately, none of those libraries seams to be complete and trustful.</p>
  
  <p><strong>Doing it myself based on a node.js low level tokenizer library</strong></p>
  
  <p>I can do it my self with a low level tokenizer library like:</p>
  
  <ul>
  <li><a href="http://jison.org" rel="noreferrer">jison</a> (1,457 stars on <a href="https://github.com/zaach/jison" rel="noreferrer">github</a>, 240 daily downloads on <a href="https://www.npmjs.org/package/jison" rel="noreferrer">npm</a>)</li>
  <li><a href="https://github.com/floby/node-tokenizer" rel="noreferrer">tokenizer</a> (44 stars on <a href="https://github.com/Floby/node-tokenizer" rel="noreferrer">github</a>, 10 daily downloads on <a href="https://www.npmjs.org/package/tokenizer" rel="noreferrer">npm</a>)</li>
  </ul>
  
  <p><strong>Doing it myself based on existing Javascript code beautifier</strong></p>
  
  <p><a href="http://codemirror.net/" rel="noreferrer">CodeMirror</a> is a pretty cool Javascript library (browser side) that can recognize SQL keywords, strings, etc. Check ou the <a href="http://codemirror.net/1/contrib/sql/" rel="noreferrer">demo</a>.</p>
  
  <p>I can build a node.js library tokenizer based on CodeMirror.
  The <a href="https://github.com/marijnh/CodeMirror/tree/master/mode/sql" rel="noreferrer">SQL mode is here on github</a>, I can maybe adapt it to get tokens within a node application.</p>
  
  <p>PS: <a href="http://codemirror.net/" rel="noreferrer">CodeMirror</a> have 5,046 stars on <a href="https://github.com/marijnh/codemirror" rel="noreferrer">github</a> and is well maintained.</p>
  
  <hr>
  
  <p>I figured out that there are 2 distinct problems: Tokenization and Syntax validation (which is related to tokenization).</p>
  
  <p>I made myself a <strong>SQL tokenizer</strong> for Node.js based on the <a href="https://github.com/marijnh/CodeMirror/tree/master/mode/sql" rel="noreferrer">SQL mode</a> of the excellent <a href="http://codemirror.net/" rel="noreferrer">CodeMirror</a> (5,046 stars on github, well maintained). CodeMirror's SQL mode take in charge "generic" SQL and some SQL particularities like <a href="http://fr.wikipedia.org/wiki/Microsoft_SQL_Server" rel="noreferrer">MSSQL</a>, <a href="http://www.mysql.fr/" rel="noreferrer">MySQL</a>, <a href="http://fr.wikipedia.org/wiki/PL/SQL" rel="noreferrer">PL/SQL</a>, <a href="http://cassandra.apache.org/" rel="noreferrer">Cassandra</a>, Hive and <a href="https://mariadb.org/" rel="noreferrer">MariaDB</a>.</p>
  
  <p>When my project will be mature enough, I will (probably) put it public on GitHub and let you know.</p>
  
  <p>About the <strong>SQL syntax</strong> validation, I found no JavaScript tool (or open source project to adapt in JS) yet... </p>