title: Python tools for template animation / unobtrusive server scripting?
body: <p>Are there good Python libraries that provide an elegant solution for what is called "template animation", "unobtrusive server scripting" or "binding data to markup"?  </p>
  
  <p>In other words, I'm looking for a good API to bind data to plain HTML to produce a dynamic web page.</p>
  
  <p>Bonus points if it compiles the API calls and HTML into HTML-emitting code, since its slow to parse HTML, manipulate it and then serialise back to HTML.</p>
  
  <h2>Background</h2>
  
  <p>This question is prompted by the "<a href="http://www.workingsoftware.com.au/page/Your_templating_engine_sucks_and_everything_you_have_ever_written_is_spaghetti_code_yes_you" rel="nofollow">Your templating engine sucks and everything you have ever written is spaghetti code (yes, you)</a>" blog post which advocates template animation of plain HTML to avoid the mixing of languages inherent to templating systems.  </p>
  
  <p>Instead of mixing data-binding and markup in a single template (view.tmpl), separate the data-binding to its own code file (view.py), leaving only HTML in the template (view.html).  </p>
  
  <p>The clear benefit of the split is designers can write views in HTML independently of the developers and back-end implementation.  The designer can then iteratively improve the view HTML in his chosen toolset.  Designers aren't generally happy to hack away in a template language and run a full dev environment and web server to visualise the changes.  </p>
  
  <h2>Potential drawbacks of animating HTML</h2>
  
  <ul>
  <li><p><strong>Slow performance</strong>.  Parse HTML, manipulate the structure to bind data, then serialise back to HTML, easily 3x slower than processing a compiled template.  Template languages after all are really syntactic sugar for code that prints HTML.  The hquery Ruby library solves this by compiling the hquery code and HTML into traditional template.  Great demonstration of the power of Ruby-based DSLs.  Can it be done in Python?</p></li>
  <li><p><strong>Limited to HTML and XML</strong>.  Template animation relies on the structure of HTML, in constrast to many templating languages don't care about the markup is and will just as happily generate plain text or CSS as they will HTML. I don't think this is an issue for web apps, and some templating languages like Genshi pride themselves on working within the structure of valid XML.</p></li>
  <li><p><strong>Too complex</strong>.   Manipulating HTML with DOM is downright painful. Hopefully the right API will make manipulating HTML in code just as easy as adding template instructions to the HTML.</p></li>
  </ul>
  
  <h2>Investigation</h2>
  
  <h3>non-Python solutions</h3>
  
  <ul>
  <li><p><a href="https://github.com/choonkeat/hquery" rel="nofollow">hquery</a> for Ruby, described as "unobtrusive server scripting", is a DSL that fills HTML from data structures.  These HTML manipulations compile to an ERB template for fast execution.  </p></li>
  <li><p><a href="http://code.google.com/p/querytemplates/" rel="nofollow">queryTemplates</a> for PHP, described as "DOM and CSS driven templating" providing "data injection methods".  API uses jQuery-inspired call chaining.  The code describes how to bind a PHP data structure to the HTML, and compiles the result to a .php template to accept the concrete data structure</p></li>
  <li><p><a href="http://beebole.com/pure/" rel="nofollow">pure</a> JavaScript library for binding JSON to HTML.  Requires additional "directive" JSON describing how to relate the JSON data to HTML elements.     </p></li>
  <li><p><a href="http://diazo.org/" rel="nofollow">diazo</a> for Python.  Requires a rules.xml which together with the designer's plain HTML theme is compiled to an XSLT "template" that binds dynamic HTML or XML from the webapp to the theme.  </p></li>
  </ul>
  
  <p>queryTemplates, pure and diazo require an abstract description of how to bind data to HTML - in the form of chained API calls, JSON, and XML rules respectively.  The system then compiles a template to accept concrete data for binding.  The abstract data-binding description adds a layer of indirection which I suspect require more mental gymnastics to use than adding template directives to HTML.</p>
  
  <p>hquery however sets itself apart by having its API calls accept data lookup expressions that are somehow left unevaluated and retained in the compiled ERB.  It's a lisp-like "code as data" mode of operation that is at least very cool and is probably easier to write.</p>
  
  <h3>Other interesting libraries</h3>
  
  <ul>
  <li><p><a href="http://entrian.com/PyMeld/" rel="nofollow">PyMeld</a> for Python, a DOM-alike based on string replacement.  I tried it and found it cumbersome.  Filling a table involves cutting the sample row from the table, then repeatedly populating the row with record data and appending a copy to the table.  Ugh.</p></li>
  <li><p><a href="https://github.com/flatiron/plates" rel="nofollow">plates</a> for Node.js "binds data to markup" (namely, JSON to HTML).  It's like <a href="http://mustache.github.com/mustache.5.html" rel="nofollow">Mustache</a> without any Mustache template syntax.  I expect the direct filling of HTML from JSON limits it very simple data binding.  Didn't even see a looping example.</p></li>
  <li><p><a href="http://search.cpan.org/~flora/HTML-Zoom-0.009006/lib/HTML/Zoom.pm" rel="nofollow">HTML::Zoom</a> for Pearl.  It's jQuery for Perl.  Doesn't compile down.</p></li>
  </ul>
  
  <h3>Starting points for Python</h3>
  
  <ul>
  <li><p><a href="http://lxml.de/" rel="nofollow">lxml.etree</a> the de-facto Python XML/HTML library.  Maybe half as painful as DOM.  Was written for parsing/generation of XML, not data binding to existing HTML.  But it's a decent foundation on which to build a template animation library.</p></li>
  <li><p><a href="http://pypi.python.org/pypi/pyquery" rel="nofollow">pyquery</a> built on top of lxml.etree and lxml.cssquery. Emulates the jQuery API method-by-method.  This makes it superior for HTML manipulation, but lacks convenience functions for binding data structures to the HTML.  These can be added, only draw back would be slowness because its inherently a parse-manipulate-serialise process.</p></li>
  <li><p><a href="https://bitbucket.org/brandizzi/golem/overview" rel="nofollow">golem</a> by brandizzi, built on top of lxml.etree and lxml.cssquery.  It's just starting out, but a procedural API specifically for data binding.</p></li>
  </ul>
  
  <p>Even with a good API for binding data to HTML, it will be slow unless it can be compiled into HTML-emitting code, or at least into a traditional templating language.</p>
  
  <h2>Handling includes and masters</h2>
  
  <p>Handling inclusion of shared footers or navigation, and handling layout masters (aka template inheritance) is a related problem.  The  plain HTML needs to be complete and expanded to be usable as a preview, so those expansions happen before the data binding step.  Tools exist to manage common includes and  layouts - DreamWeaver for one has a library system for shared footers and the like.</p>