title: Storing Image Data for offline web application (client-side storage database)
body: <p>I have an offline web application using appcaching. I need to provide it about 10MB - 20MB of data that it will save (client-side) consisting mainly of PNG image files. The operation is as follows:</p>
  <ol>
  <li>Web application downloads and installs in appcache (uses manifest)</li>
  <li>Web app requests from server PNG data files (how? - see alternatives below)</li>
  <li>Occasionally web app resyncs with server, and does small partial updates/deletes/additions to PNG database</li>
  <li>FYI: Server is a JSON REST server, that can place files in wwwroot for pickup</li>
  </ol>
  <p>Here is my current analysis of client-based &quot;databases&quot; that handle binary blob storage</p>
  <h1>SEE UPDATE at Bottom</h1>
  <ul>
  <li><p><strong>AppCache</strong> (via manifest add all the PNG and then update on demand)</p>
  <ul>
  <li>CON: any change of a PNG database item will mean complete download of all items in manifest (Really bad news!)</li>
  </ul>
  </li>
  <li><p><strong>WebStorage</strong></p>
  <ul>
  <li>CON: Designed for JSON storage</li>
  <li>CON: can only store blobs via base64 encoding (probably fatal flaw due to cost of de-encoding)</li>
  <li>CON: Hard limit of 5MB for webStorage <a href="http://htmlui.com/blog/2011-08-23-5-obscure-facts-about-html5-localstorage.html" rel="noreferrer">http://htmlui.com/blog/2011-08-23-5-obscure-facts-about-html5-localstorage.html</a></li>
  </ul>
  </li>
  <li><p><strong>PhoneGap &amp; SQLLite</strong></p>
  <ul>
  <li>CON: Sponsor will reject it as a native app requiring certification</li>
  </ul>
  </li>
  <li><p><strong>ZIP file</strong></p>
  <ul>
  <li>Server creates a zip file, places it in wwwroot, and notifies client</li>
  <li>user has to manually unzip (At least that is how I see it) and save to client file system</li>
  <li>Web app uses FileSystem API to reference files</li>
  <li>CON: ZIP might be too large (zip64?), long time to create</li>
  <li>CON: Not sure if FileSystem API can always read out of the sandbox (I think so)</li>
  </ul>
  </li>
  <li><p><strong>USB or SD card</strong> (back to the stone age....)</p>
  <ul>
  <li>The user will be local to the server before going offline</li>
  <li>So we could have him insert a SD card, let the server fill it with PNG files</li>
  <li>Then the user will plug it into the laptop, tablet</li>
  <li>Web app will use FileSystem API to read the files</li>
  <li>CON: Not sure if FileSystem API can always read out of the sandbox (I think so)</li>
  </ul>
  </li>
  <li><p><strong>WebSQL</strong></p>
  <ul>
  <li>CON: w3c has abandoned it (pretty bad)</li>
  <li>I might consider a Javascript wrapper that uses IndexedDB and WebSQL as a fall-back</li>
  </ul>
  </li>
  <li><p><strong>FileSystem API</strong></p>
  <ul>
  <li>Chrome supports read/write of blobs</li>
  <li>CON: not clear about IE and FireFox (IE10, has non-standard msSave)</li>
  <li>caniuse.com reports IOS and Android support (but again, is this just r/w of JSON, or does it include the full blob API for writing?</li>
  <li>CON: FireFox folks dislike FileSystem API &amp; not clear if they are supporting saving blobs: <a href="https://hacks.mozilla.org/2012/07/why-no-filesystem-api-in-firefox/" rel="noreferrer">https://hacks.mozilla.org/2012/07/why-no-filesystem-api-in-firefox/</a></li>
  <li>PRO: <em>Much</em> faster than IndexedDB for blobs according to jsperf <a href="http://jsperf.com/indexeddb-vs-localstorage/15" rel="noreferrer">http://jsperf.com/indexeddb-vs-localstorage/15</a> (page 2)</li>
  </ul>
  </li>
  <li><p><strong>IndexedDB</strong></p>
  <ul>
  <li>Good support in IE10, FireFox (save, read blobs)</li>
  <li>Good speed and easier management than a file system (deletes, updates)</li>
  <li>PRO: see speed tests: <a href="http://jsperf.com/indexeddb-vs-localstorage/15" rel="noreferrer">http://jsperf.com/indexeddb-vs-localstorage/15</a></li>
  <li>See this article on storing and display of images in IndexedDB: <a href="https://hacks.mozilla.org/2012/02/storing-images-and-files-in-indexeddb/" rel="noreferrer">https://hacks.mozilla.org/2012/02/storing-images-and-files-in-indexeddb/</a></li>
  <li>CON: I confirmed that Chrome does not yet support blob writing (current bug, but not clear when it will be fixed)</li>
  <li>UPDATE: <a href="https://developers.google.com/web/updates/2014/07/Blob-support-for-IndexedDB-landed-on-Chrome-Dev" rel="noreferrer">A June 2014 blogpost</a> suggests Chrome now supports blobs in <code>IndexedDB</code></li>
  <li>UPDATE: <a href="https://caniuse.com/#search=indexeddb" rel="noreferrer">This caniuse/indexeddb</a> confirms: &quot;Chrome 36 and below did not support Blob objects as indexedDB values.&quot;; suggesting &gt;Chrome36 supports Blob objects.</li>
  </ul>
  </li>
  <li><p><strong>LawnChair</strong> JavaScript wrapper <a href="http://brian.io/lawnchair/" rel="noreferrer">http://brian.io/lawnchair/</a></p>
  <ul>
  <li>PRO: very clean wrapper for IndexedDB, WebSQL or whatever database you have (think polyfill)</li>
  <li>CON: cannot store binary blobs, only data:uri (base64 encoding) (probably fatal flaw due to cost of de-encoding)</li>
  </ul>
  </li>
  <li><p><strong>IndexedDB JQUERY</strong> polyFill <a href="https://github.com/axemclion/jquery-indexeddb" rel="noreferrer">https://github.com/axemclion/jquery-indexeddb</a></p>
  <ul>
  <li>Parashuram has writtent a nice JQUERY wrapper for the raw IndexedDB interface</li>
  <li>PRO: greatly simplifies using IndexedDB, I was hoping to add a shim/polyfill for Chrome FileSystemAPI</li>
  <li>CON: It should handle blobs, but I was unable to get it to work</li>
  </ul>
  </li>
  <li><p><strong>idb.filesystem.js</strong> <a href="http://ericbidelman.tumblr.com/post/21649963613/idb-filesystem-js-bringing-the-html5-filesystem-api" rel="noreferrer">http://ericbidelman.tumblr.com/post/21649963613/idb-filesystem-js-bringing-the-html5-filesystem-api</a></p>
  <ul>
  <li>Eric Bidelman @ Google has written a well tested PolyFill the FileSystem API that uses Indexed DB as a fall back</li>
  <li>PRO: FileSystem API is well suited for storing blobs</li>
  <li>PRO: works great on FireFox and Chrome
  <ul>
  <li>PRO: great for synchronizing with cloud based CouchDB</li>
  </ul>
  </li>
  <li>CON: no clear why, but it is not working on IE10</li>
  </ul>
  </li>
  <li><p><strong>PouchDB</strong> JavaScript Library <a href="http://pouchdb.com/" rel="noreferrer">http://pouchdb.com/</a></p>
  <ul>
  <li>great for syncing a CouchDB with a local DB (uses either WebSQL or IndexedDB (not my problem though)</li>
  <li>CON: NO CONS, PouchDB now supports binary blobs for all recent browsers (IE, Chrome, Firefox, Chrome on mobile, etc.) as well as many older browsers. That was not the case when I first did this post.</li>
  </ul>
  </li>
  </ul>
  <p>NOTE: to see a data:uri encoding of PNG I created an example at: <a href="http://jsbin.com/ivefak/1/edit" rel="noreferrer">http://jsbin.com/ivefak/1/edit</a></p>
  <p><strong>Desired/Usefull/Uneeded Features</strong></p>
  <ul>
  <li>No native (EXE, PhoneGap, ObjectiveC, etc) app on client (pure web application)</li>
  <li>Only needs to run on latest Chrome, FireFox, IE10 for laptops</li>
  <li>Strongly want same solution for Android Tablet (IOS would be nice too) but only need one browser to work (FF, Chrome, etc.)</li>
  <li>Fast initial DB population</li>
  <li>REQUIREMENT: Very fast retrieval of images by web application from storage (DB, file)</li>
  <li>Not meant for consumers. We can restrict browsers, and ask user to do special setup &amp; tasks, but let's minimize that</li>
  </ul>
  <p><strong>IndexedDB Implementations</strong></p>
  <ul>
  <li>There is an excellent article on how IE,FF,and Chrome internally implement this at: <a href="http://www.aaron-powell.com/web/indexeddb-storage" rel="noreferrer">http://www.aaron-powell.com/web/indexeddb-storage</a></li>
  <li>In short:
  <ul>
  <li>IE uses the same database format as Exchange and Active Directory for IndexedDB</li>
  <li>Firefox is using SQLite so are kind of implementing a NoSQL database in to SQL database</li>
  <li>Chrome (and WebKit) are using a Key/ Value store which has heritage in BigTable</li>
  </ul>
  </li>
  </ul>
  <p><strong>My Current Results</strong></p>
  <ul>
  <li>I chose to use an IndexedDB approach (and polyfill with FileSystemAPI for Chrome until they ship blob support)</li>
  <li>For fetching the tiles, I had a dilemna since the JQUERY folks are kvetching about adding this to AJAX</li>
  <li>I went with XHR2-Lib by Phil Parsons, which is very much like JQUERY .ajax() <a href="https://github.com/p-m-p/xhr2-lib" rel="noreferrer">https://github.com/p-m-p/xhr2-lib</a></li>
  <li>Performance for 100MB downloads (IE10 4s, Chrome 6s, FireFox 7s).</li>
  <li>I could not get any of the IndexedDB wrappers to work for blobs (lawnchair, PouchDB, jquery-indexeddb, etc.)</li>
  <li>I rolled my own wrapper, and performance is (IE10 2s, Chrome 3s, FireFox 10s)</li>
  <li>With FF, I assume we are seeing the performance issue of using a relational DB (sqllite) for a non-sql storage</li>
  <li>NOTE, Chrome has outstanding debug tools (developer tab, resources) for inspecting the state of the IndexedDB.</li>
  </ul>
  <p><strong>FINAL Results posted below as answer</strong></p>
  <h1>Update</h1>
  <p>PouchDB now supports binary blobs for all recent browsers (IE, Chrome, Firefox, Chrome on mobile, etc.) as well as many older browsers. That was not the case when I first did this post.</p>