text: # -*- coding: utf-8 -*-
  #@+leo-ver=5-thin
  #@+node:EKR.20040517080250.1: * @file mod_http.py
  #@@first
  #@+<< docstring >>
  #@+node:ekr.20050111111238: ** << docstring >>
  #@@language rest
  #@@wrap
  '''An http plug-in for LEO, based on AsyncHttpServer.py.
  
  Adapted and extended from the Python Cookbook:
  http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/259148
  
  This plug-in has three distinct behaviors:
  
      - Viewing loaded outlines in a browser, e.g. at http://localhost:8130/
      - Storing web bookmarks in Leo outlines from a browser, e.g. at
        http://localhost:8130/_/add/bkmk/
      - Remotely executing code in a running Leo instance, e.g. at
        http://localhost:8130/_/exec/?cmd=nd=p.insertAsLastChild()
  
  Install this plug-in is as follows:
  
  Start Leo with the plug-in enabled. You will see a purple message that says
  something like::
  
      "http serving enabled at 127.0.0.1:8130"
  
  Settings
  --------
  
  ``@bool http_active = True``
      required for plug-in to be active
  ``@string http_ip = 127.0.0.1``
      address to bind to, see notes below
  ``@int  http_port = 8130``
      port to use (1 3 0 ~= L E O)
  ``@bool http_allow_remote_exec = False``
      must be changed to True for remote code execution
  ``@string rst_http_attributename = 'rst_http_attribute'``
      link to obsolete rst3 plugin
  ``@data user_bookmark_stylesheet``
      Additional .css for bookmarks.
  ``@data http_stylesheet``
      The default .css for this page.
  ``@data user_http_stylesheet``
      Additional .css for this page.
  
  **Note**: The html generated by the server contains both stylesheets
  as inline <style> elements, with the user_http_stylesheet contents last.
  
  ``@data mod_http script``
      The body text of this @data setting contains *all* the
      javascript used in the page.
  
  **Note** The html generated by the server handles the <script> elements::
  
      <script src="http://ajax.googleapis.com/ajax/libs/jquery/1.11.2/jquery.min.js">
      </script>
      <script>..the contents of @data mod_http_script..
      </script>
  
  Browsing Leo files
  ------------------
  
  Start a web browser, and enter the following URL: http://localhost:8130/
  
  You will see a a "top" level page containing one link for every open .leo file.
  Start clicking :-)
  
  You can use the browser's refresh button to update the top-level view in the
  browser after you have opened or closed files.
  
  **Note**: IP address 127.0.0.1 is accessible by all users logged into your
  local machine. That means while Leo and mod_http is running anyone logged into
  your machine will be able to browse all your leo outlines and add bookmarks.
  
  **Note**: If you want all other network accessible machines to have access
  to your mod_http instance, then use @string http_ip = 0.0.0.0.
  
  **Note**: the browser_encoding constant (defined in the top node of this file)
  must match the character encoding used in the browser. If it does not, non-ascii
  characters will look strange.
  
  Saving bookmarks from browser to Leo
  ------------------------------------
  
  To do this, add a bookmark to the browser with the following URL / Location::
  
      javascript:w=window; d=w.document; ln=[];if(w.location.href.indexOf('one-tab')>-1){el=d.querySelectorAll('a');for (i in el){ln.push({url:el[i].href,txt:el[i].innerHTML});};};w.open('http://localhost:8130/_/add/bkmk/?&name=' + escape(d.title) + '&selection=' + escape(window.getSelection()) + '&ln=' + escape(JSON.stringify(ln)) + '&url=' + escape(w.location.href),"_blank","toolbar=no, location=no, directories=no, status=no, menubar=no, scrollbars=no, resizable=yes, copyhistory=no, width=800, height=300, status=no");void(0); # NOQA
  
  and edit the port (8130 in the example above) to match the port you're using for
  mod_http.
  
  Bookmarks are created as the first node in the outline which has been opened
  longest. You can set the ``@string`` ``http_bookmark_unl`` to specify an
  alternative location, e.g.::
  
      @string http_bookmark_unl = /home/tbrown/.bookmarks.leo#@bookmarks-->Incoming
  
  to place them in the `Incoming` node in the `@bookmarks` node in the
  `.bookmarks.leo` outline.
  
  The headline is preceded with '@url ' *unless* the ``bookmarks`` plug-in is
  loaded. If the ``bookmarks`` plug-in is loaded the bookmark will have to be moved
  to a ``@bookmarks`` tree to be useful.
  
  **Note**: there is special support for Chrome's OneTab extension as a mechanism
  for saving all tabs open. Click the OneTab button to get a list of tabs, then
  click the "Share all as web page" link to show the list on www.one-tab.com.
  Bookmark as usual as described above. You can then delete the shared list with
  the "Delete this shared page" button. The Leo node created will have a child
  node for each of the listed tabs.
  
  The browser may or may not be able to close the bookmark form window for you,
  depending on settings - set ``dom.allow_scripts_to_close_windows`` to true in
  ``about:config`` in Firefox.
  
  Executing code remotely
  -----------------------
  
  **Warning**:
  
      Allowing remote code execution is a **HUGE SECURITY HOLE**, you need to
      be sure that the url from which you access Leo (typically
      http://localhost:8130/) is accessible only by people and software you trust.
  
      Remote execution is turned off by default, you need to manually / locally
      change the @setting ``@bool http_allow_remote_exec = False`` to ``True``
      to enable it.
  
  Commands to be executed are submitted via HTTP GET requests, which can
  be generated in almost any language and also triggered from shortcuts,
  links in other documents or applications, etc. etc.
  
  The basic form is::
  
      http://localhost:8130/_/exec/?cmd=<python code for Leo to execute>
  
  The query parameters are:
  
  ``cmd`` (required)
      A valid python snippet for Leo to execute. Executed by the ``eval``
      command in the ``mod_scripting`` plug-in. Can be specified multiple times, each
      is executed in order. May contain newlines, see examples.
  
  ``c`` (optional)
      Which currently loaded outline to use, can be an integer, starting from
      zero, or the full path+filename, or just the base filename. Defaults to 0
      (zero), i.e. the "first" open outline.
  
  ``enc`` (optional)
      Encoding for response, 'str', 'repr', or 'json'. Used to render the returned
      value.
  
  ``mime_type`` (optional)
      Defaults to ``text/plain``. Could be useful to use ``text/html`` etc.
  
  A special variant url is::
  
      http://localhost:8130/_/exec/commanders/
  
  which returns a list of open outlines.
  
  Examples
  ========
  
  This command::
  
      curl http://localhost:8130/_/exec/?cmd='c.bringToFront()' >/dev/null
  
  will raise the Leo window, or at least make the window manager signal the need
  to raise it.
  
  ::
  
      curl --get --data-urlencode \
        cmd='g.handleUrl("file:///home/tbrown/.leo/.contacts.leo#Contacts", c)' \
        http://localhost:8130/_/exec/ >/dev/null
  
  will cause a running Leo instance to open ``/some/path/contacts.leo`` and select
  the ``Contacts`` node.  A desktop icon link, browser bookmark, or link in a
  spread-sheet or other document could be used the same way.
  
  In the ``bash`` shell language, this code::
  
      TEXT="$@"
      curl --silent --show-error --get --data-urlencode cmd="
          nd = c.rootPosition().insertAfter()
          nd.h = 'TODO: $TEXT'
          import time
          nd.b = '# created %s' % time.asctime()
          c.selectPosition(nd)
          c.redraw()
          'To do item created\n'
      " http://localhost:8130/_/exec/
  
  could be written in a file called ``td``, and then, assuming that file is
  executable and on the shell's path, entering::
  
      td remember to vacuum the cat
  
  on the command line would create a node at the top of the first open outline in
  Leo with a headline ``TODO: remember to vacuum the cat`` and a body text ``#
  created Wed Jul 29 16:42:26 2015``. The command ``vs-eval`` returns the value of
  the last expression in a block, so the trailing ``'To do item created\n'`` gives
  better feedback than ``None`` generated by ``c.redraw()``.
  ``c.selectPosition(nd)`` is important ant to stop Leo getting confused about
  which node is selected.
  
  
  '''
  #@-<< docstring >>
  #@+<< imports >>
  #@+node:EKR.20040517080250.3: ** << imports >>
  # pylint: disable=deprecated-method
      # parse_qs
  import leo.core.leoGlobals as g
  import asynchat
  import asyncore
  import json
  import http.server
  SimpleHTTPRequestHandler = http.server.SimpleHTTPRequestHandler
  import io
  StringIO = io.StringIO
  BytesIO = io.BytesIO
  import urllib
  import urllib.parse as urlparse
  import os
  import select
  import shutil
  import socket
  import time
  from xml.sax.saxutils import quoteattr
  #@-<< imports >>
  #@+<< data >>
  #@+node:ekr.20161001100345.1: ** << data >>
  browser_encoding = 'utf-8' # To do: Can we query the browser for this?
      # This encoding must match the character encoding used in your browser.
      # If it does not, non-ascii characters will look very strange.
  
  sockets_to_close = []
  #@-<< data >>
  #@+others
  #@+node:ekr.20060830091349: ** init & helpers (mod_http.py)
  def init():
      '''Return True if the plugin has loaded successfully.'''
      if 0:
          g.registerHandler("open2", onFileOpen)
      else:
          getGlobalConfiguration()
          if config.http_active:
              try:
                  Server(config.http_ip, config.http_port, RequestHandler)
              except socket.error as e:
                  g.es("mod_http server initialization failed (%s:%s): %s" % (
                      config.http_ip, config.http_port, e))
                  return False
              asyncore.read = a_read
              g.registerHandler("idle", plugin_wrapper)
              g.es("http serving enabled at %s:%s" % (
                  config.http_ip, config.http_port), color="purple")
      g.plugin_signon(__name__)
      return True
  #@+node:tbrown.20111005140148.18223: *3* getGlobalConfiguration
  def getGlobalConfiguration():
      """read config."""
      # timeout.
      newtimeout = g.app.config.getInt("http-timeout")
      if newtimeout is not None:
          config.http_timeout = newtimeout / 1000.0
      # ip.
      newip = g.app.config.getString("http-ip")
      if newip:
          config.http_ip = newip
      # port.
      newport = g.app.config.getInt("http-port")
      if newport:
          config.http_port = newport
      # active.
      newactive = g.app.config.getBool("http-active")
      if newactive is not None:
          config.http_active = newactive
      # attribute name.
      new_rst2_http_attributename = g.app.config.getString("rst2-http-attributename")
      if new_rst2_http_attributename:
          config.rst2_http_attributename = new_rst2_http_attributename
  #@+node:EKR.20040517080250.45: *3* plugin_wrapper
  def plugin_wrapper(tag, keywords):
      if g.app.killed:
          return
      # first = True
      while loop(config.http_timeout):
          pass
  #@+node:bwmulder.20050326191345.1: *3* onFileOpen (not used) (mod_http.py)
  def onFileOpen(tag, keywords):
      c = keywords.get("new_c")
      g.trace('c',repr(c))
      wasactive = config.http_active
      getConfiguration(c)
      if config.http_active and not wasactive: # Ok for unit testing:
          Server('', config.http_port, RequestHandler)
          asyncore.read = a_read
          g.registerHandler("idle", plugin_wrapper)
          g.es("http serving enabled on port %s, " % (
              config.http_port),
              color="purple")
  #@+node:EKR.20040517080250.48: *3* getConfiguration (not used)
  def getConfiguration(c):
      """Called when the user opens a new file."""
      # timeout.
      newtimeout = c.config.getInt("http-timeout")
      if newtimeout is not None:
          config.http_timeout = newtimeout / 1000.0
      # port.
      newport = c.config.getInt("http-port")
      if newport:
          config.http_port = newport
      # active.
      newactive = c.config.getBool("http-active")
      if newactive is not None:
          config.http_active = newactive
      # attribute name.
      new_rst2_http_attributename = c.config.getString("rst2-http-attributename")
      if new_rst2_http_attributename:
          config.rst2_http_attributename = new_rst2_http_attributename
  #@+node:ekr.20161003140938.1: ** getData
  def getData(setting):
      '''Return the given @data node.'''
      # Plug an important security hole.
      c = g.app and g.app.log and g.app.log.c
      key = g.app.config.munge(setting)
      if c and key == 'httpscript' and c.config.isLocalSetting(key, 'data'):
          g.issueSecurityWarning('@data http-script')
          return ""
      aList = g.app.config.getData(
          key,
          strip_comments=False,
          strip_data=False,
      )
      s = ''.join(aList or [])
      return s
  #@+node:bwmulder.20050326191345: ** class config
  class config:
      enabled = None # True when security check re http-allow-remote-exec passes.
      http_active = False
      http_timeout = 0
      http_ip = '127.0.0.1'
      http_port = 8130
      rst2_http_attributename = 'rst_http_attribute'
  #@+node:EKR.20040517080250.4: ** class delayedSocketStream
  class delayedSocketStream(asyncore.dispatcher_with_send):
      #@+others
      #@+node:EKR.20040517080250.5: *3* __init__
      def __init__(self, sock):
          # pylint: disable=super-init-not-called
          self._map = asyncore.socket_map
          self.socket = sock
          self.socket.setblocking(0)
          self.closed = 1 # compatibility with SocketServer
          self.buffer = []
      #@+node:EKR.20040517080250.6: *3* write
      def write(self, data):
          self.buffer.append(data)
      #@+node:EKR.20040517080250.7: *3* initiate_sending
      def initiate_sending(self):
          # Create a bytes string.
          aList = [g.toEncodedString(z) for z in self.buffer]
          self.out_buffer = b''.join(aList)
          del self.buffer
          self.set_socket(self.socket, None)
          self.socket.setblocking(0)
          self.connected = 1
          try:
              self.addr = self.socket.getpeername()
          except socket.error:
              # The addr isn't crucial
              pass
      #@+node:EKR.20040517080250.8: *3* handle_read
      def handle_read(self):
          pass
      #@+node:EKR.20040517080250.9: *3* writable
      def writable(self):
          result = (not self.connected) or len(self.out_buffer)
          if not result:
              sockets_to_close.append(self)
          return result
      #@-others
  #@+node:EKR.20040517080250.20: ** class leo_interface
  class leo_interface:
      # pylint: disable=no-member
          # .path, .send_error, .send_response and .end_headers
          # appear to be undefined.
      #@+others
      #@+node:bwmulder.20050322224921: *3* send_head & helpers
      def send_head(self):
          """Common code for GET and HEAD commands.
  
           This sends the response code and MIME headers.
  
           Return value is either a file object (which has to be copied
           to the outputfile by the caller unless the command was HEAD,
           and must be closed by the caller under all circumstances), or
           None, in which case the caller has nothing further to do.
  
           """
          try:
              # self.path is provided by the RequestHandler class.
              path = self.split_leo_path(self.path)
              if path[0] == '_':
                  f = self.leo_actions.get_response()
              elif len(path) == 1 and path[0] == 'favicon.ico':
                  f = self.leo_actions.get_favicon()
              elif path == '/':
                  f = self.write_leo_windowlist()
              else:
                  try:
                      window, root = self.find_window_and_root(path)
                      if window is None:
                          self.send_error(404, "File not found")
                          return None
                      if root is None:
                          self.send_error(404, "No root node")
                          return None
                      f = StringIO()
                      self.write_leo_tree(f, window, root)
                  except nodeNotFound:
                      self.send_error(404, "Node not found")
                      return None
                  except noLeoNodePath:
                      g.es("No Leo node path:", path)
                      # Is there something better we can do here?
                      self.send_error(404, "Node not found")
                      return None
              if f is None:
                  return None
              length = f.tell()
              f.seek(0)
              self.send_response(200)
              self.send_header("Content-type", getattr(f, "mime_type", "text/html"))
              self.send_header("Content-Length", str(length))
              self.end_headers()
              return f
          except Exception:
              import traceback
              traceback.print_exc()
              raise
          return None
      #@+node:EKR.20040517080250.26: *4* find_window_and_root
      def find_window_and_root(self, path):
          """
          given a path of the form:
              [<short filename>,<number1>,<number2>...<numbern>]
              identify the leo node which is in that file, and,
              from top to bottom, is the <number1> child of the topmost
              node, the <number2> child of that node, and so on.
  
              Return None if that node can not be identified that way.
          """
          for w in g.app.windowList:
              if w.shortFileName() == path[0]:
                  return w, w.c.rootPosition()
          return None, None
      #@+node:EKR.20040517080250.30: *4* split_leo_path
      def split_leo_path(self, path):
          '''Split self.path.'''
          if path == '/':
              return '/'
          if path.startswith("/"):
              path = path[1:]
          return path.split('/')
      #@+node:ekr.20161001114512.1: *4* write_leo_tree & helpers
      def write_leo_tree(self, f, window, root):
          '''Wriite the entire html file to f.'''
          root = root.copy()
          self.write_head(f, root.h, window)
          f.write('<body>')
          f.write('<div class="container">')
          f.write('<div class="outlinepane">')
          f.write('<h1>%s</h1>' % window.shortFileName())
          for sib in root.self_and_siblings():
              self.write_node_and_subtree(f, sib)
          f.write('</div>')
          f.write('</div>')
          self.write_body_pane(f, root)
          f.write('</body></html>')
      #@+node:ekr.20161001124752.1: *5* write_body_pane
      def write_body_pane(self, f, p):
  
          f.write('<div class="bodypane">')
          f.write('<pre class="body-text">')
          f.write('<code class="body-code">%s</code>' % escape(p.b))
              # This isn't correct when put in a triple string.
              # We might be able to use g.adjustTripleString, but this works.
          f.write('</pre></div>')
      #@+node:ekr.20161001121838.1: *5* write_head
      def write_head(self, f, headString, window):
  
          f.write("""<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
      "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
      <html>
      <head>
          <meta http-equiv="content-type" content="text/html; charset=UTF-8" />
          <style>%s</style>
          <style>%s</style>
          <script src="http://ajax.googleapis.com/ajax/libs/jquery/1.11.2/jquery.min.js">
          </script>
          <script>%s</script>
          <title>%s</title>
      </head>
      """ % (
              getData('http_stylesheet'),
              getData('user_http_stylesheet'),
              getData('http_script'),
              (escape(window.shortFileName() + ":" + headString)))
          )
      #@+node:ekr.20161001122919.1: *5* write_node_and_subtree
      def write_node_and_subtree(self, f, p):
  
          # This organization, with <headline> elements in <node> elements,
          # allows proper highlighting of nodes.
          f.write('<div class="node" id=n:%s>' % (
              quoteattr(p.gnx),
          ))
          f.write('<div class="headline" id=h:%s expand="%s" icon="%02d" b=%s>%s</div>' % (
              quoteattr(p.gnx),
              '+' if p.hasChildren() else '-',
              p.computeIcon(),
              quoteattr(p.b),
              escape(p.h),
          ))
          for child in p.children():
              self.write_node_and_subtree(f, child)
          f.write('</div>')
      #@+node:EKR.20040517080250.27: *4* write_leo_windowlist
      def write_leo_windowlist(self):
          f = StringIO()
          f.write('''<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
      "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
      <html>
      <head>
          <meta http-equiv="content-type" content="text/html; charset=UTF-8" />
          <style>%s</style>
          <style>%s</style>
          <title>ROOT for LEO HTTP plugin</title>
      </head>
      <body>
          <h1>Windowlist</h1>
          <hr />
          <ul>
      ''' % (
          getData('http_stylesheet'),
          getData('user_http_stylesheet'),
      ))
          a = g.app # get the singleton application instance.
          windows = a.windowList # get the list of all open frames.
          for w in windows:
              shortfilename = w.shortFileName()
              f.write('<li><a href="%s">"file name: %s"</a></li>' % (
                  shortfilename, shortfilename))
          f.write('</ul><hr /></body></html>')
          return f
      #@+node:bwmulder.20050319135316: *3* node_reference & helpers
      def node_reference(self, vnode):
          """
          Given a position p, return the name of the node.
  
          This is called from leo.core.leoRst.
          """
          # 1. Find the root
          root = vnode
          parent = root.parent()
          while parent:
              root = parent
              parent = root.parent()
          while root.v._back:
              root.moveToBack()
          # 2. Return the window
          window = [w for w in g.app.windowList if w.c.rootVnode().v == root.v][0]
          result = self.create_leo_h_reference(window, vnode)
          return result
      #@+node:EKR.20040517080250.21: *4* add_leo_links
      def add_leo_links(self, window, node, f):
          """
          Given a node 'node', add links to:
              The next sibling, if any.
              the next node.
              the parent.
              The children, if any.
          """
          # Collecting the navigational links.
          if node:
              nodename = node.h
              threadNext = node.threadNext()
              sibling = node.next()
              parent = node.parent()
              f.write("<p>\n")
              children = []
              firstChild = node.firstChild()
              if firstChild:
                  child = firstChild
                  while child:
                      children.append(child)
                      child = child.next()
              if threadNext is not None:
                  self.create_leo_reference(window, threadNext, "next", f)
              f.write("<br />")
              if sibling is not None:
                  self.create_leo_reference(window, sibling, "next Sibling", f)
              f.write("<br />")
              if parent is None:
                  self.create_href("/", "Top level", f)
              else:
                  self.create_leo_reference(window, parent, "Up", f)
              f.write("<br />")
              f.write("\n</p>\n")
          else:
              # top level
              child = window.c.rootPosition()
              children = [child]
              next = child.next()
              while next:
                  child = next
                  children.append(child)
                  next = child.next()
              nodename = window.shortFileName()
          if children:
              f.write("\n<h2>")
              f.write("Children of ")
              f.write(escape(nodename))
              f.write("</h2>\n")
              f.write("<ol>\n")
              for child in children:
                  f.write("<li>\n")
                  self.create_leo_reference(window, child, child.h, f)
                  f.write("</li>\n")
              f.write("</ol>\n")
      #@+node:EKR.20040517080250.22: *4* create_href
      def create_href(self, href, text, f):
          f.write('<a href="%s">' % href)
          f.write(escape(text))
          f.write("</a>\n")
      #@+node:bwmulder.20050319134815: *4* create_leo_h_reference
      def create_leo_h_reference(self, window, node):
          parts = [window.shortFileName()] + self.get_leo_nameparts(node)
          href = '/' + '/'.join(parts)
          return href
      #@+node:EKR.20040517080250.23: *4* create_leo_reference
      def create_leo_reference(self, window, node, text, f):
          """
          Create a reference to 'node' in 'window', displaying 'text'
          """
          href = self.create_leo_h_reference(window, node)
          self.create_href(href, text, f)
      #@+node:EKR.20040517080250.28: *4* write_path
      def write_path(self, node, f):
          result = []
          while node:
              result.append(node.h)
              node = node.parent()
          result.reverse()
          if result:
              result2 = result[: -1]
              if result2:
                  result2 = ' / '.join(result2)
                  f.write("<p>\n")
                  f.write("<br />\n")
                  f.write(escape(result2))
                  f.write("<br />\n")
                  f.write("</p>\n")
              f.write("<h2>")
              f.write(escape(result[-1]))
              f.write("</h2>\n")
      #@-others
  #@+node:tbrown.20110930093028.34530: ** class LeoActions
  class LeoActions:
      """
      A place to collect other URL based actions like saving bookmarks from
      the browser. Conceptually this stuff could go in class leo_interface
      but putting it here for separation for now.
      """
      #@+others
      #@+node:tbrown.20110930220448.18077: *3* __init__(LeoActions)
      def __init__(self, request_handler):
          self.request_handler = request_handler
          self.bookmark_unl = g.app.commanders()[0].config.getString('http-bookmark-unl')
          self.exec_handler = ExecHandler(request_handler)
      #@+node:tbrown.20110930220448.18075: *3* add_bookmark
      def add_bookmark(self):
          """Return the file like 'f' that leo_interface.send_head makes
  
          """
          parsed_url = urlparse.urlparse(self.request_handler.path)
          query = urlparse.parse_qs(parsed_url.query)
          # print(parsed_url.query)
          # print(query)
          name = query.get('name', ['NO TITLE'])[0]
          url = query['url'][0]
          one_tab_links = []
          if 'www.one-tab.com' in url.lower():
              one_tab_links = query.get('ln', [''])[0]
              one_tab_links = json.loads(one_tab_links)
          c = None # outline for bookmarks
          previous = None # previous bookmark for adding selections
          parent = None # parent node for new bookmarks
          using_root = False
          path = self.bookmark_unl
          if path:
              parsed = urlparse.urlparse(path)
              leo_path = os.path.expanduser(parsed.path)
              c = g.openWithFileName(leo_path, old_c=None)
              if c:
                  g.es_print("Opened '%s' for bookmarks" % path)
                  if parsed.fragment:
                      g.recursiveUNLSearch(parsed.fragment.split("-->"), c)
                  parent = c.currentPosition()
                  if parent.hasChildren():
                      previous = parent.getFirstChild()
              else:
                  g.es_print("Failed to open '%s' for bookmarks" % self.bookmark_unl)
          if c is None:
              using_root = True
              c = g.app.commanders()[0]
              parent = c.rootPosition()
              previous = c.rootPosition()
          f = StringIO()
          if previous and url == previous.b.split('\n', 1)[0]:
              # another marking of the same page, just add selection
              self.add_bookmark_selection(
                  previous, query.get('selection', [''])[0])
              c.selectPosition(previous) # required for body text redraw
              c.redraw()
              f.write("""
      <body onload="setTimeout('window.close();', 350);" style='font-family:mono'>
      <p>Selection added</p></body>"""         )
              return f
          if '_form' in query:
              # got extra details, save to new node
              f.write("""
      <body onload="setTimeout('window.close();', 350);" style='font-family:mono'>
      <p>Bookmark saved</p></body>"""         )
              if using_root:
                  nd = parent.insertAfter()
                  nd.moveToRoot(c.rootPosition())
              else:
                  nd = parent.insertAsNthChild(0)
              if g.pluginIsLoaded('leo.plugins.bookmarks'):
                  nd.h = name
              else:
                  nd.h = '@url ' + name
              selection = query.get('selection', [''])[0]
              if selection:
                  selection = '\n\n"""\n' + selection + '\n"""'
              tags = query.get('tags', [''])[0]
              if one_tab_links:
                  if tags:
                      tags += ', OneTabList'
                  else:
                      tags = 'OneTabList'
                  self.get_one_tab(one_tab_links, nd)
              nd.b = "%s\n\nTags: %s\n\n%s\n\nCollected: %s%s\n\n%s" % (
                  url,
                  tags,
                  query.get('_name', [''])[0],
                  time.strftime("%c"),
                  selection,
                  query.get('description', [''])[0],
              )
              c.setChanged()
              c.selectPosition(nd) # required for body text redraw
              c.redraw()
              return f
          # send form to collect extra details
          f.write("""
      <html>
      <head>
          <style>
              body {font-family:mono; font-size: 80%%;}
              th {text-align:right}
          </style>
          <style>%s</style>
      <title>Leo Add Bookmark</title>
      </head>
      <body onload='document.getElementById("tags").focus();'>
          <form method='GET' action='/_/add/bkmk/'>
              <input type='hidden' name='_form' value='1'/>
              <input type='hidden' name='_name' value=%s/>
              <input type='hidden' name='selection' value=%s/>
              <input type='hidden' name='ln' value=%s/>
              <table>
              <tr><th>Tags:</th><td><input id='tags' name='tags' size='60'/>(comma sep.)</td></tr>
              <tr><th>Title:</th><td><input name='name' value=%s size='60'/></td></tr>
              <tr><th>URL:</th><td><input name='url' value=%s size='60'/></td></tr>
              <tr><th>Notes:</th><td><textarea name='description' cols='60' rows='6'></textarea></td></tr>
              </table>
              <input type='submit' value='Save'/><br/>
          </form>
      </body>
      </html>""" % (
              getData('user_bookmark_stylesheet'),  # EKR: Add config.css to style.
              quoteattr(name),
              quoteattr(query.get('selection', [''])[0]),
              quoteattr(json.dumps(one_tab_links)),
              quoteattr(name),
              quoteattr(url)))
          return f
      #@+node:tbrown.20131122091143.54044: *3* get_one_tab
      def get_one_tab(self, links, nd):
          """get_one_tab - Add child bookmarks from OneTab chrome extension
  
          :Parameters:
          - `links`: list of {'txt':, 'url':} dicts
          - `nd`: node under which to put child nodes
          """
          for link in links:
              if 'url' in link and 'www.one-tab.com' not in link['url'].lower():
                  nnd = nd.insertAsLastChild()
                  nnd.h = link['txt']
                  nnd.b = "%s\n\nTags: %s\n\n%s\n\nCollected: %s%s\n\n%s" % (
                      link['url'],
                      'OneTabTab',
                      link['txt'],
                      time.strftime("%c"),
                      '',
                      '',
                  )
      #@+node:tbrown.20111002082827.18325: *3* add_bookmark_selection
      def add_bookmark_selection(self, node, text):
          '''Insert the selected text into the bookmark node,
          after any earlier selections but before the users comments.
  
              http://example.com/
  
              Tags: tags, are here
  
              Full title of the page
  
              Collected: timestamp
  
              """
              The first saved selection
              """
  
              """
              The second saved selection
              """
  
              Users comments
  
          i.e. just above the "Users comments" line.
          '''
          b = node.b.split('\n')
          insert = ['', '"""', text, '"""']
          collected = None
          tri_quotes = []
          for n, i in enumerate(b):
              if collected is None and i.startswith('Collected: '):
                  collected = n
              if i == '"""':
                  tri_quotes.append(n)
          if collected is None:
              # not a regularly formatted text, just append
              b.extend(insert)
          elif len(tri_quotes) >= 2:
              # insert after the last balanced pair of tri quotes
              x = tri_quotes[len(tri_quotes) - len(tri_quotes) % 2 - 1] + 1
              b[x: x] = insert
          else:
              # found Collected but no tri quotes
              b[collected + 1: collected + 1] = insert
          node.b = '\n'.join(b)
          node.setDirty()
      #@+node:tbrown.20111005093154.17683: *3* get_favicon
      def get_favicon(self):
          path = g.os_path_join(g.computeLeoDir(), 'Icons', 'LeoApp16.ico')
          try:
              f = StringIO()
              f2 = open(path)
              s = f2.read()
              f.write(s)
              return f
          except Exception:
              return None
      #@+node:tbrown.20110930220448.18076: *3* get_response
      def get_response(self):
          """Return the file like 'f' that leo_interface.send_head makes"""
          if self.request_handler.path.startswith('/_/add/bkmk/'):
              return self.add_bookmark()
          if self.request_handler.path.startswith('/_/exec/'):
              return self.exec_handler.get_response()
          f = StringIO()
          f.write("Unknown URL in LeoActions.get_response()")
          return f
      #@-others
  #@+node:tbrown.20150729112701.1: ** class ExecHandler
  class ExecHandler:
      """
      Quasi-RPC GET based interface
      """
      #@+others
      #@+node:tbrown.20150729112701.2: *3* __init__
      def __init__(self, request_handler):
          self.request_handler = request_handler
      #@+node:tbrown.20150729112808.1: *3* get_response
      def get_response(self):
          """Return the file like 'f' that leo_interface.send_head makes"""
          # self.request_handler.path.startswith('/_/exec/')
  
          if not g.app.config.getBool("http-allow-remote-exec"):
              return None  # fail deliberately
              
          c = g.app and g.app.log and g.app.log.c
          if c and config.enable is None:
              if c.config.isLocalSetting('http-allow-remote-exec', 'bool'):
                  g.issueSecurityWarning('@bool http-allow-remote-exec')
                  config.enable = False
              else:
                  config.enable = True
  
          parsed_url = urlparse.urlparse(self.request_handler.path)
          query = urlparse.parse_qs(parsed_url.query)
  
          enc = query.get("enc", ["str"])[0]
  
          if parsed_url.path.startswith('/_/exec/commanders/'):
              ans = [i.fileName() for i in g.app.commanders()]
              if enc != 'json':
                  ans = '\n'.join(ans)
          else:
              ans = self.proc_cmds()
  
          f = StringIO()
          f.mime_type = query.get("mime_type", ["text/plain"])[0]
          enc = query.get("enc", ["str"])[0]
          if enc == 'json':
              f.write(json.dumps(ans))
          elif enc == 'repr':
              f.write(repr(ans))
          else:
              f.write(str(ans))
          return f
  
      #@+node:tbrown.20150729150843.1: *3* proc_cmds (mod_http.py)
      def proc_cmds(self):
  
          parsed_url = urlparse.urlparse(self.request_handler.path)
          query = urlparse.parse_qs(parsed_url.query)
          # work out which commander to use, zero index int, full path name, or file name
          c_idx = query.get('c', [0])[0]
          # pylint: disable=literal-comparison
          if c_idx != 0:
              try:
                  c_idx = int(c_idx)
              except ValueError:
                  paths = [i.fileName() for i in g.app.commanders()]
                  if c_idx in paths:
                      c_idx = paths.index(c_idx)
                  else:
                      paths = [os.path.basename(i) for i in paths]
                      c_idx = paths.index(c_idx)
          ans = None
          c = g.app.commanders()[c_idx]
          if c and c.evalController:
              for cmd in query['cmd']:
                  ans = c.evalController.eval_text(cmd)
          return ans  # the last answer, if multiple commands run
      #@-others
  #@+node:EKR.20040517080250.10: ** class nodeNotFound
  class nodeNotFound(Exception):
      pass
  #@+node:bwmulder.20061014153544: ** class noLeoNodePath
  class noLeoNodePath(Exception):
      """
      Raised if the path can not be converted a filename and a series of numbers.
      Most likely a reference to a picture.
      """
      pass
  #@+node:EKR.20040517080250.13: ** class RequestHandler
  class RequestHandler(
      leo_interface,
      asynchat.async_chat,
      SimpleHTTPRequestHandler
  ):
      # pylint: disable=too-many-ancestors
      # pylint: disable=super-init-not-called
      #@+others
      #@+node:EKR.20040517080250.14: *3* __init__
      def __init__(self, conn, addr, server):
          self.leo_actions = LeoActions(self)
          super().__init__(conn)
          self.client_address = addr
          self.connection = conn
          self.server = server
          self.wfile = delayedSocketStream(self.socket)
          # Sets the terminator. When it is received, this means that the
          # http request is complete, control will be passed to self.found_terminator
          self.term = g.toEncodedString('\r\n\r\n')
          self.set_terminator(self.term)
          self.buffer = BytesIO()
          # Set self.use_encoding and self.encoding.
          # This is used by asyn_chat.
          self.use_encoding = True
          self.encoding = 'utf-8'
      #@+node:EKR.20040517080250.15: *3* copyfile
      def copyfile(self, source, outputfile):
          """Copy all data between two file objects.
  
          The SOURCE argument is a file object open for reading
          (or anything with a read() method) and the DESTINATION
          argument is a file object open for writing (or
          anything with a write() method).
  
          The only reason for overriding this would be to change
          the block size or perhaps to replace newlines by CRLF
          -- note however that this the default server uses this
          to copy binary data as well.
           """
          shutil.copyfileobj(source, outputfile, length=255)
      #@+node:EKR.20040517080250.16: *3* log_message
      def log_message(self, format, *args):
          """Log an arbitrary message.
  
           This is used by all other logging functions.  Override
           it if you have specific logging wishes.
  
           The first argument, FORMAT, is a format string for the
           message to be logged.  If the format string contains
           any % escapes requiring parameters, they should be
           specified as subsequent arguments (it's just like
           printf!).
  
           The client host and current date/time are prefixed to
           every message.
  
           """
          message = "%s - - [%s] %s\n" % (
              self.address_string(),
              self.log_date_time_string(),
              format % args)
          g.es(message)
      #@+node:EKR.20040517080250.17: *3* collect_incoming_data
      def collect_incoming_data(self, data):
          """Collects the data arriving on the connexion"""
          self.buffer.write(data)
      #@+node:EKR.20040517080250.18: *3* prepare_POST
      def prepare_POST(self):
          """Prepare to read the request body"""
          bytesToRead = int(self.headers.getheader('content-length'))
          # set terminator to length (will read bytesToRead bytes)
          self.set_terminator(bytesToRead)
          self.buffer = StringIO()
          # control will be passed to a new found_terminator
          self.found_terminator = self.handle_post_data
      #@+node:EKR.20040517080250.19: *3* handle_post_data
      def handle_post_data(self):
          """Called when a POST request body has been read"""
          self.rfile = StringIO(self.buffer.getvalue())
          self.do_POST()
          self.finish()
      #@+node:EKR.20040517080250.31: *3* do_GET
      def do_GET(self):
          """Begins serving a GET request"""
          # nothing more to do before handle_data()
          self.handle_data()
      #@+node:EKR.20040517080250.32: *3* do_POST
      def do_POST(self):
          """
          Begins serving a POST request. The request data must be readable
          on a file-like object called self.rfile
          """
          # pylint: disable=undefined-variable
              # urlparse_header and urlparse_multipart appear to be undefined.
          ctype, pdict = urlparse_header(self.headers.getheader('content-type'))
          length = int(self.headers.getheader('content-length'))
          if ctype == 'multipart/form-data':
              query = urlparse_multipart(self.rfile, pdict)
          elif ctype == 'application/x-www-form-urlencoded':
              qs = self.rfile.read(length)
              query = urllib.parse.parse_qs(qs, keep_blank_values=1)
          else:
              query = '' # Unknown content-type
          # some browsers send 2 more bytes...
          [ready_to_read, x, y] = select.select([self.connection], [], [], 0)
          if ready_to_read:
              self.rfile.read(2)
          self.QUERY.update(self.query(query))
          self.handle_data()
      #@+node:EKR.20040517080250.33: *3* query
      def query(self, parsedQuery):
          """Returns the QUERY dictionary, similar to the result of urllib.parse_qs
           except that :
           - if the key ends with [], returns the value (a Python list)
           - if not, returns a string, empty if the list is empty, or with the
           first value in the list"""
          res = {}
          for item in parsedQuery.keys():
              value = parsedQuery[item] # a Python list
              if item.endswith("[]"):
                  res[item[: -2]] = value
              else:
                  res[item] = value[0] if value else ''
          return res
      #@+node:EKR.20040517080250.34: *3* handle_data
      def handle_data(self):
          """Class to override"""
          f = self.send_head()
          if f:
              self.copyfile(f, self.wfile)
      #@+node:ekr.20110522152535.18254: *3* handle_read_event (NEW)
      def handle_read_event(self):
          '''Over-ride SimpleHTTPRequestHandler.handle_read_event.'''
          asynchat.async_chat.handle_read_event(self)
      #@+node:EKR.20040517080250.35: *3* handle_request_line (aka found_terminator)
      def handle_request_line(self):
          """Called when the http request line and headers have been received"""
          # prepare attributes needed in parse_request()
          self.rfile = BytesIO(self.buffer.getvalue())
          self.raw_requestline = self.rfile.readline()
          self.parse_request()
          # if there is a Query String, decodes it in a QUERY dictionary
          self.path_without_qs, self.qs = self.path, ''
          if self.path.find('?') >= 0:
              self.qs = self.path[self.path.find('?') + 1:]
              self.path_without_qs = self.path[: self.path.find('?')]
          self.QUERY = self.query(urlparse.parse_qs(self.qs, 1))
          if self.command in ['GET', 'HEAD']:
              # if method is GET or HEAD, call do_GET or do_HEAD and finish
              method = "do_" + self.command
              if hasattr(self, method):
                  f = getattr(self, method)
                  f()
                  self.finish()
          elif self.command == "POST":
              # if method is POST, call prepare_POST, don't finish before
              self.prepare_POST()
          else:
              self.send_error(501, "Unsupported method (%s)" % self.command)
      #@+node:ekr.20110522152535.18256: *3* found_terminator
      def found_terminator(self):
          # pylint: disable=method-hidden
          # Control may be passed to another found_terminator.
          self.handle_request_line()
      #@+node:EKR.20040517080250.36: *3* finish
      def finish(self):
          """Reset terminator (required after POST method), then close"""
          self.set_terminator(self.term)
          self.wfile.initiate_sending()
          # self.close()
      #@-others
  #@+node:EKR.20040517080250.37: ** class Server
  class Server(asyncore.dispatcher):
      """Copied from http_server in medusa"""
      #@+others
      #@+node:EKR.20040517080250.38: *3* __init__
      def __init__(self, ip, port, handler):
          self.ip = ip
          self.port = port
          self.handler = handler
          super().__init__()
          self.create_socket(socket.AF_INET, socket.SOCK_STREAM)
          self.set_reuse_addr()
          self.bind((ip, port))
          # lower this to 5 if your OS complains
          self.listen(1024)
      #@+node:EKR.20040517080250.39: *3* handle_accept
      def handle_accept(self):
          try:
              # pylint: disable=unpacking-non-sequence
              # The following except statements catch this.
              conn, addr = self.accept()
          except socket.error:
              self.log_info('warning: server accept() threw an exception', 'warning')
              return
          except TypeError:
              self.log_info('warning: server accept() threw EWOULDBLOCK', 'warning')
              return
          # creates an instance of the handler class to handle the request/response
          # on the incoming connexion
          self.handler(conn, addr, self)
      #@-others
  #@+node:ekr.20140920145803.17997: ** functions
  #@+node:EKR.20040517080250.47: *3* a_read (asynchore override)
  def a_read(obj):
      try:
          obj.handle_read_event()
      except asyncore.ExitNow:
          raise
      except Exception:
          obj.handle_error()
  #@+node:ekr.20110522152535.18252: *3* escape
  def escape(s):
      s = s.replace('&', "&amp;")
      s = s.replace('<', "&lt;")
      s = s.replace('>', "&gt;")
      # is there a more elegant way to do this?
      # Replaces blanks with &nbsp; id they are in
      # the beginning of the line.
      lines = s.split('\n')
      result = []
      blank = chr(32)
      for line in lines:
          if line.startswith(blank):
              resultchars = []
              startline = True
              for char in line:
                  if char == blank:
                      if startline:
                          resultchars.append('&nbsp;')
                      else:
                          resultchars.append(' ')
                  else:
                      startline = False
                      resultchars.append(char)
              result.append(''.join(resultchars))
          else:
              result.append(line)
      s = '\n'.join(result)
      s = s.replace('\n', '<br />')
      s = s.replace(chr(9), '&nbsp;&nbsp;&nbsp;&nbsp;')
      # 8/9/2007
      # s = g.toEncodedString(s,encoding=browser_encoding,reportErrors=False)
      # StringIO.write(self, s)
      return s
  #@+node:EKR.20040517080250.44: *3* loop (asynchore override)
  def loop(timeout=5.0, use_poll=0, map=None):
      """
      Override the loop function of asynchore.
      We poll only until there is not read or
      write request pending.
      """
      return poll(timeout)
  #@+node:bwmulder.20050322135114: *3* node_reference
  def node_reference(vnode):
      """
      Use by the rst3 plugin.
      """
      return leo_interface().node_reference(vnode)
  #@+node:EKR.20040517080250.40: *3* poll
  def poll(timeout=0.0):
      global sockets_to_close
      map = asyncore.socket_map
      if not map:
          return False
      while 1:
          r = []; w = []; e = []
          for fd, obj in map.items():
              if obj.readable():
                  r.append(fd)
              if obj.writable():
                  w.append(fd)
          if not sockets_to_close: # Set by writeable()
              break
          for s in sockets_to_close:
              s.close()
          sockets_to_close = []
      if [] == r == w == e:
          time.sleep(timeout)
      else:
          #@+<< try r, w, e = select.select >>
          #@+node:EKR.20040517080250.41: *4* << try r, w, e = select.select >>
          try:
              r, w, e = select.select(r, w, e, timeout)
          except select.error: # as err:
              # if err[0] != EINTR:
                  # raise
              # else:
                  # return False
              return False # EKR: EINTR is undefined.
          #@-<< try r, w, e = select.select >>
      for fd in r:
          #@+<< asyncore.read(map.get(fd)) >>
          #@+node:EKR.20040517080250.42: *4* << asyncore.read(map.get(fd)) >>
          obj = map.get(fd)
          if obj is not None:
              asyncore.read(obj)
          #@-<< asyncore.read(map.get(fd)) >>
      for fd in w:
          #@+<< asyncore.write(map.get(fd)) >>
          #@+node:EKR.20040517080250.43: *4* << asyncore.write(map.get(fd)) >>
          obj = map.get(fd)
          if obj is not None:
              asyncore.write(obj)
          #@-<< asyncore.write(map.get(fd)) >>
      return len(r) > 0 or len(w) > 0
  #@+node:bwmulder.20050322132919: *3* rst_related functions
  #@+node:bwmulder.20050322132919.2: *4* get_http_attribute
  def get_http_attribute(p):
      if hasattr(p.v, 'unknownAttributes'):
          return p.v.unknownAttributes.get(config.rst2_http_attributename, None)
      return None
  #@+node:bwmulder.20050322134325: *4* reconstruct_html_from_attrs
  def reconstruct_html_from_attrs(attrs, how_much_to_ignore=0):
      """
      Given an attribute, reconstruct the html for this node.
      """
      result = []
      stack = attrs
      while stack:
          result.append(stack[0])
          stack = stack[2]
      result.reverse()
      result = result[how_much_to_ignore:]
      result.extend(attrs[3:])
      stack = attrs
      for i in range(how_much_to_ignore):
          stack = stack[2]
      while stack:
          result.append(stack[1])
          stack = stack[2]
      return result
  #@+node:bwmulder.20050322133050: *4* set_http_attribute
  def set_http_attribute(p, value):
      vnode = p.v
      if hasattr(vnode, 'unknownAttributes'):
          vnode.unknownAttributes[config.rst2_http_attributename] = value
      else:
          vnode.unknownAttributes = {config.rst2_http_attributename: value}
  #@-others
  #@@language python
  #@@tabwidth -4
  #@-leo
  