Refactor: export lines instead of ticks.

Timeline axis in CoffeeScript
=============================

Introduction and description
----------------------------

This demo is aimed at exploring possibilities of visualizing time axis.  This especially applies to situations where time axis is meant to be used in interactive plotting, being able to change from very short time intervals to very long time intervals.

Main points of usability of the timeline are:
 * Area that the timeline takes (ideally, as little as possible).
 * Clarity and readability (ideally text/info should be concise and distinctive).
 * How much information is showed (ideally, as much as current zoom level and resolution can afford).
 * Should be easily navigable (should work also for visually impaired).
 * Should be interactive (ideally react according to its representation).

Test cases
----------

### Visual test cases

 1. All time labels and ticks should correspond to their times
 2. Daylight-saving time should be displayed correctly
 3. Leap seconds should be displayed properly
 4. Every interval type should be displayed properly
   1. year
   2. month
   3. week
   4. day
   5. hour
   6. minute
   7. second
   8. millisecond
 5. Check that lines and text are sharp
 6. There should be no intersection of features

### Unit test cases

To be done:
 * Labels should not intersect with each other
 * There should be at least one label (or no?)
 * Label should be in range if its coordinates are in viewport
 * Check for daylight-saving time
 * Check for daylight-saving time table for several historical intervals
 * Check for leap seconds

Development concerns
--------------------

### Readability

The following things should be thought of before making a final version:
 * DPI should be taken into account.  It should work fine on any DPI display.
 * Every element should be separated from others.
    * Visual elements should not intersect with each other.
    * Visual elements such as labels and ticks should be at some good distance to each other.
 * It is a good idea to alternate colors of intervals.
    * Use two colors.
    * They should differ for visually impaired.
    * And should be suitable for grey-color display.
    * Thus, they should differ in brightness at least.
 * Continuos change (opposed to discrete) between scales (that is zoom levels).
    * Makes it possible to animate without jumps between representation levels, keeping viewer connection to details.
    * Adds more information to viewer.  A question is how to keep this simple enough and not bloated with information and graphics.

### Portability

To make it more flexible to render (to canvas, svg, vega, anything else), we should have an intermediate object as output of axis formatting, which could be rendered after withe a chosen rendeder.

Library functions
-----------------

### The __TimelineMaker__ class

Since formatting and rendering of timeline involves a lot of calculations, it is divided in several functions, and they are combined in the __TimelineMaker__ class.  The __options__ parameter will be a dictionary with values that are needed for formatting the timeline.

    class TimelineMaker
      constructor: (@options) ->

The two functions that are intended to be called are __formatTimeAxis()__, which formats time axis into a dictionary that describes the look of axis, and one of the __renderTo...__ functions that render that data dictionary to a desired context.

#### The __formatTimeAxis()__ function

A function that formats time axis into an intermediate format.  It has two parameters, __interval__ (a dictionary with __start__ and __end__ values, each of Date type) and corresponding to that interval __width__ of a viewport.  It returns a formatted time axis object.  This object is a collection of features with their coordinates in a viewport (the top left point of the viewport is (0,0), the top-right is (0, width)).

      formatTimeAxis: (interval, width) ->
        {@start, end} = interval
        @intervalLength = end - @start
        @width = width

We can put on axis ticks and labels, and also colour areas between them.  They correspond to a set of time points.  Labels could correspond to time points or to time intervals between these points.

Ticks should correspond to _edge points_ of time, a point of time between two days (00-00), two years, months, weeks, or a sharp time point, having integer number of hours, or minutes, or seconds.  In general, while moving from bigger to smaller time interval types, every interval type has to be in integer amount, until  some point.  For example (10 years, 2 months, 3 days) from Epoch and remainder which is less than one day.  That means whe should have a parameter that corresponds to the smallest time interval that has to be integral.  We call this parameter _options.intervalType_.  We also need a number of this intervals between each tick.  This parameter will be __options.intervalMultiplier__.  There is one exception though:  in case of weeks there is no previous integral interval.  So there are two cases:  week and (year, month, day, hour, minute, second, millisecond).

So we need to build a list of time points that correspond to a given time interval.  We will put such code in a special function, __findPointList()__.

        pointList = @findPointList @start, end

Now, when we have found the list of time points, we need to construct a dictionary with graphical element properties.  To transform time value into a coordinate we use the __timeToCoord()__ function.  We start with ticks.  Each tick is a line.  The @options.tickLength parameter is a base length of a tick.  We assume that (y = 0) is a baseline and tick length is from baseline to `@options.tickLength` down and `@options.tickLength/5` up.

We should probably change ticks dictionary to lines dictionary instead, so we can add other types of lines, like baseline.  It could be better also to move all constants to options and make ticks drawing more flexible.

        ticks = for timePoint in pointList
          {
            x1: @timeToCoord timePoint
            x2: @timeToCoord timePoint
            y1: -@options.tickLength / 5
            y2: @options.tickLength
          }

Now we add text labels too.

Here we also need to improve formatting, now it's just a quick fix to display text.  Text should be formatted without problems on any display and resolution and shouldn't intersect ticks when it has reasonable font size.

        textLabels = for timePoint in pointList
          {
            x: @timeToCoord timePoint
            y: @options.tickLength * 1.2
            text: switch @options.intervalType
              when 'year' then timePoint.getFullYear().toString()
              when 'month' then timePoint.getMonth().toString()
              when 'week', 'day' then timePoint.getDate().toString()
              when 'hour' then timePoint.getHours().toString()
              when 'minute' then timePoint.getMinutes().toString()
              when 'second' then timePoint.getSeconds().toString()
              when 'millisecond' then timePoint.getMilliseconds().toString()
          }

Now we combine all elements into a one dictionary and return it.

        {lines: ticks, textLabels}

#### The __findPointList()__ function

This function returns a list of points that are needed to be calculated for a (__start__, __end__) interval.

      findPointList: (start, end) ->

Since we could need labels between time points, we need time points inside the interval (non-inclusive) and one point on left and right side.  We can build the point list by finding the leftmost _edge point_ which is strictly less than the __start__ of the interval.  We move code that does that to the __findLeftTime()__.

        timePoint = @findLeftTime start

Then, we populate the list in by incrementing points until the we reach right end of the interval, using __findNextPoint()__ function.

        pointList = [timePoint]
        until timePoint > end
          timePoint = @findNextPoint timePoint
          pointList.push timePoint
        pointList

#### The __findLeftTime()__ function

The __findLeftTime()__ function calculates the rightmost point of time for current __options.intervalType__ such that it is not inside the given __interval__ (excluding beginning).  The __options.intervalType__ could be one of the following:
 * 'year'
 * 'month'
 * 'week'
 * 'day'
 * 'hour'
 * 'minute'
 * 'second'
 * 'millisecond'
Another option, __options.intervalMultiplier__, says how many of such intervals are between two time points.  It should be an integer value.  For years and months we truncate them to the desired value, while for days and weeks we will count from a fixed origin day near Epoch (Monday 5 january 1970), so that any day intervals are independent from underlying months and years.  That also means that in case of months the __options.intervalMultiplier__ should be equal to 1, 2, 3, 4 or 6 to be displayed correctly.  To calculate number of days or weeks (reduced to 7 calculation for 7 days) from origin day we use binary subtracting, starting from huge multiplier equal to 1048576 (roughly 2800/11200 years), going down to base multiplier equal to 1.  For interval types smaller or equal than hours we use the __findNextPoint()__ function by incrementing local origin (the interval type and everything smaller is reset to 0).  This is done to avoid problems with daylight-saving and similar things.  In __findNextPoint()__ we make sure that edge points are consistent while using different values of __start__.

      findLeftTime: (start) ->
        leftTime = new Date start.getTime()
        switch @options.intervalType
          when 'year'
            newYear = start.getFullYear() - start.getFullYear() % @options.intervalMultiplier
            leftTime.setFullYear newYear, 0, 1
            leftTime.setHours 0, 0, 0, 0
          when 'month'
            newMonth = start.getMonth() - start.getMonth() % @options.intervalMultiplier
            leftTime.setMonth newMonth, 1
            leftTime.setHours 0, 0, 0, 0
          when 'week', 'day'
            leftTime.setFullYear 1970, 0, 5
            leftTime.setHours 0, 0, 0, 0
            delta = 1048576
            intervalMultiplier = @options.intervalMultiplier
            if @options.intervalType is 'week'
              intervalMultiplier *= 7
            while delta >= 1
              nextTime = new Date leftTime.getFullYear(),
                leftTime.getMonth()
                leftTime.getDate() + delta*intervalMultiplier
                leftTime.getHours()
                leftTime.getMinutes()
                leftTime.getSeconds()
                leftTime.getMilliseconds()
              delta /= 2
              leftTime = nextTime if nextTime <= start
          when 'hour'
            leftTime.setHours 0, 0, 0, 0
            while leftTime < start
              nextTime = @findNextPoint leftTime
              leftTime = nextTime if nextTime <= start
          when 'minute'
            newMinutes = start.getMinutes() - start.getMinutes() % @options.intervalMultiplier
            leftTime.setMinutes newMinutes, 0, 0
          when 'second'
            leftTime.setSeconds start.getSeconds(), 0
            while leftTime < start
              nextTime = @findNextPoint leftTime
              leftTime = nextTime if nextTime <= start
          when 'millisecond'
            newMilliseconds = start.getMilliseconds() - start.getMilliseconds() % @options.intervalMultiplier
            leftTime.setMilliseconds newMilliseconds
        leftTime

There is a code repetition in the switch statement above.  One can think of a good way to improve this part of code.

### The __findNextPoint()__ function

This function calculates next edge time point for a current options assuming __timePoint__ argument is an edge time point.  We take care of daylight-saving time and leap seconds here, assuming that at most one hour/minute/second/millisecond is added or subtracted.  Also we assume that subtraction is not happening at time with 0 value, that is something like 00 -> 23 is not happening.

In current implementation we cope with daylight-saving time by adding hours in UTC to date and then checking how the value changes in local time and fixing by one hour if needed.  The same for leap seconds.

      findNextPoint: (timePoint) ->
        nextTime = new Date timePoint.getTime()
        switch @options.intervalType
          when 'year'
            nextTime.setFullYear (timePoint.getFullYear() + @options.intervalMultiplier)
          when 'month'
            nextTime.setMonth (timePoint.getMonth() + @options.intervalMultiplier)
          when 'week'
            nextTime.setDate (timePoint.getDate() + 7*@options.intervalMultiplier)
          when 'day'
            nextTime.setDate (timePoint.getDate() + @options.intervalMultiplier)
          when 'hour'
            nextTime.setUTCHours (timePoint.getUTCHours() + @options.intervalMultiplier)
            hours = nextTime.getHours()
            if hours % @options.intervalMultiplier isnt 0
              if (hours+1) % @options.intervalMultiplier is 0
                nextTime.setUTCHours (nextTime.getUTCHours() + 1)
              else
                nextTime.setUTCHours (nextTime.getUTCHours() - 1)
          when 'minute'
            nextTime.setMinutes (timePoint.getMinutes() + @options.intervalMultiplier)
          when 'second'
            nextTime.setUTCSeconds (timePoint.getUTCSeconds() + @options.intervalMultiplier)
            seconds = nextTime.getSeconds()
            if seconds % @options.intervalMultiplier isnt 0
              if (seconds+1) % @options.intervalMultiplier is 0
                nextTime.setUTCSeconds (nextTime.getUTCSeconds() + 1)
              else
                nextTime.setUTCSeconds (nextTime.getUTCSeconds() - 1)
          when 'millisecond'
            nextTime.setUTCMilliseconds (timePoint.getUTCMilliseconds() + @options.intervalMultiplier)
        nextTime

#### The __timeToCoord()__ function

To get coordinate of time point we use `@intervalLength` that we stored in `formatTimeAxis()`, which is equal to the number of milliseconds between `end` and `start` of the interval.  We use it to calculate pixe/time ratio, equal to `@width / @intervalLength`.

      timeToCoord: (time) ->
        timeFromStart = time - @start
        coordinate = timeFromStart * @width / @intervalLength

#### The __renderToCanvas()__ function

This function renders formatted time axis to html canvas.
 * `axisData` is formatted data (that we get by calling any of the `format...()` functions)
 * `canvas` is a html canvas object on which axis is drawn
 * `left` and `top` are x- and y-coordinates of the canvas that correspond to the (0, 0) point of the axis viewport

      renderToCanvas: (axisData, canvas, left, top) ->

In the beginning we do just regular initialization of context and its properties.  We also wrap our drawing code into `save()` and `restore()` to not alter context properties globally.

        context = canvas.getContext '2d'
        context.save()
        context.strokeStyle = '#000000'
        context.fillStyle = '#000000'
        context.lineWidth = 1
        context.font = 'normal 12px sans-serif'
        context.textAlign = 'center'
        context.textBaseline = 'top'

To all drawing functions we pass coordinates altered by `roundForCanvas()` function, which rounds values in such a way that lines are more sharp.

        for line in axisData.lines
          x1 = left + line.x1
          x2 = left + line.x2
          y1 = top + line.y1
          y2 = top + line.y2
          context.moveTo @roundForCanvas(x1), @roundForCanvas(y1)
          context.lineTo @roundForCanvas(x2), @roundForCanvas(y2)
        context.stroke()

        for textLabel in axisData.textLabels
          x = left + textLabel.x
          y = top + textLabel.y
          context.fillText textLabel.text, x, y

        context.restore()

#### The `roundForCanvas()` function

This function may be used to avoud aliasing of lines, especially on low-resolution displays.  It rounds coordinate to middle-pixel values, which are .5 in case of html canvas.

      roundForCanvas: (coord) ->
        0.5 + Math.round(coord-0.5)

Library tests
-------------

Examples
--------

This simple code displays time axis when html page is loaded, in `timeline` canvas element.

    makeDemo = ->
      canvas = document.getElementById 'timeline'
      context = canvas.getContext '2d'
      context.fillStyle = '#77FFBB'
      context.fillRect(0, 0, canvas.clientWidth, canvas.clientHeight)
      context.stroke()
      timelineMaker = new TimelineMaker {tickLength: 25, intervalType: 'day', intervalMultiplier: 1}
      start = new Date('2015-06-15T00:00:00')
      end = new Date('2015-07-13T15:23:49')
      axisData = timelineMaker.formatTimeAxis {start, end}, canvas.width
      timelineMaker.renderToCanvas axisData, canvas, 0, 15

    window.onload = makeDemo

Timeline axis in CoffeeScript
=============================

Introduction and description
----------------------------

This demo is aimed at exploring possibilities of visualizing time axis.  This especially applies to situations where time axis is meant to be used in interactive plotting, being able to change from very short time intervals to very long time intervals.

Main points of usability of the timeline are:
 * Area that the timeline takes (ideally, as little as possible).
 * Clarity and readability (ideally text/info should be concise and distinctive).
 * How much information is showed (ideally, as much as current zoom level and resolution can afford).
 * Should be easily navigable (should work also for visually impaired).
 * Should be interactive (ideally react according to its representation).

Test cases
----------

### Visual test cases

 1. All time labels and ticks should correspond to their times
 2. Daylight-saving time should be displayed correctly
 3. Leap seconds should be displayed properly
 4. Every interval type should be displayed properly
   1. year
   2. month
   3. week
   4. day
   5. hour
   6. minute
   7. second
   8. millisecond
 5. Check that lines and text are sharp
 6. There should be no intersection of features

### Unit test cases

To be done:
 * Labels should not intersect with each other
 * There should be at least one label (or no?)
 * Label should be in range if its coordinates are in viewport
 * Check for daylight-saving time
 * Check for daylight-saving time table for several historical intervals
 * Check for leap seconds

Development concerns
--------------------

### Readability

The following things should be thought of before making a final version:
 * DPI should be taken into account.  It should work fine on any DPI display.
 * Every element should be separated from others.
    * Visual elements should not intersect with each other.
    * Visual elements such as labels and ticks should be at some good distance to each other.
 * It is a good idea to alternate colors of intervals.
    * Use two colors.
    * They should differ for visually impaired.
    * And should be suitable for grey-color display.
    * Thus, they should differ in brightness at least.
 * Continuos change (opposed to discrete) between scales (that is zoom levels).
    * Makes it possible to animate without jumps between representation levels, keeping viewer connection to details.
    * Adds more information to viewer.  A question is how to keep this simple enough and not bloated with information and graphics.

### Portability

To make it more flexible to render (to canvas, svg, vega, anything else), we should have an intermediate object as output of axis formatting, which could be rendered after withe a chosen rendeder.

Library functions
-----------------

### The __TimelineMaker__ class

Since formatting and rendering of timeline involves a lot of calculations, it is divided in several functions, and they are combined in the __TimelineMaker__ class.  The __options__ parameter will be a dictionary with values that are needed for formatting the timeline.

    class TimelineMaker
      constructor: (@options) ->

The two functions that are intended to be called are __formatTimeAxis()__, which formats time axis into a dictionary that describes the look of axis, and one of the __renderTo...__ functions that render that data dictionary to a desired context.

#### The __formatTimeAxis()__ function

A function that formats time axis into an intermediate format.  It has two parameters, __interval__ (a dictionary with __start__ and __end__ values, each of Date type) and corresponding to that interval __width__ of a viewport.  It returns a formatted time axis object.  This object is a collection of features with their coordinates in a viewport (the top left point of the viewport is (0,0), the top-right is (0, width)).

      formatTimeAxis: (interval, width) ->
        {@start, end} = interval
        @intervalLength = end - @start
        @width = width

We can put on axis ticks and labels, and also colour areas between them.  They correspond to a set of time points.  Labels could correspond to time points or to time intervals between these points.

Ticks should correspond to _edge points_ of time, a point of time between two days (00-00), two years, months, weeks, or a sharp time point, having integer number of hours, or minutes, or seconds.  In general, while moving from bigger to smaller time interval types, every interval type has to be in integer amount, until  some point.  For example (10 years, 2 months, 3 days) from Epoch and remainder which is less than one day.  That means whe should have a parameter that corresponds to the smallest time interval that has to be integral.  We call this parameter _options.intervalType_.  We also need a number of this intervals between each tick.  This parameter will be __options.intervalMultiplier__.  There is one exception though:  in case of weeks there is no previous integral interval.  So there are two cases:  week and (year, month, day, hour, minute, second, millisecond).

So we need to build a list of time points that correspond to a given time interval.  We will put such code in a special function, __findPointList()__.

        pointList = @findPointList @start, end

Now, when we have found the list of time points, we need to construct a dictionary with graphical element properties.  To transform time value into a coordinate we use the __timeToCoord()__ function.  We start with ticks.  Each tick is a line.  The @options.tickLength parameter is a base length of a tick.  We assume that (y = 0) is a baseline and tick length is from baseline to `@options.tickLength` down and `@options.tickLength/5` up.

We should probably change ticks dictionary to lines dictionary instead, so we can add other types of lines, like baseline.  It could be better also to move all constants to options and make ticks drawing more flexible.

        ticks = for timePoint in pointList
          {
            x: @timeToCoord timePoint
            yTop: -@options.tickLength / 5
            yBottom: @options.tickLength
          }

Now we add text labels too.

Here we also need to improve formatting, now it's just a quick fix to display text.  Text should be formatted without problems on any display and resolution and shouldn't intersect ticks when it has reasonable font size.

        textLabels = for timePoint in pointList
          {
            x: @timeToCoord timePoint
            y: @options.tickLength * 1.2
            text: switch @options.intervalType
              when 'year' then timePoint.getFullYear().toString()
              when 'month' then timePoint.getMonth().toString()
              when 'week', 'day' then timePoint.getDate().toString()
              when 'hour' then timePoint.getHours().toString()
              when 'minute' then timePoint.getMinutes().toString()
              when 'second' then timePoint.getSeconds().toString()
              when 'millisecond' then timePoint.getMilliseconds().toString()
          }

Now we combine all elements into a one dictionary and return it.

        {ticks, textLabels}

#### The __findPointList()__ function

This function returns a list of points that are needed to be calculated for a (__start__, __end__) interval.

      findPointList: (start, end) ->

Since we could need labels between time points, we need time points inside the interval (non-inclusive) and one point on left and right side.  We can build the point list by finding the leftmost _edge point_ which is strictly less than the __start__ of the interval.  We move code that does that to the __findLeftTime()__.

        timePoint = @findLeftTime start

Then, we populate the list in by incrementing points until the we reach right end of the interval, using __findNextPoint()__ function.

        pointList = [timePoint]
        until timePoint > end
          timePoint = @findNextPoint timePoint
          pointList.push timePoint
        pointList

#### The __findLeftTime()__ function

The __findLeftTime()__ function calculates the rightmost point of time for current __options.intervalType__ such that it is not inside the given __interval__ (excluding beginning).  The __options.intervalType__ could be one of the following:
 * 'year'
 * 'month'
 * 'week'
 * 'day'
 * 'hour'
 * 'minute'
 * 'second'
 * 'millisecond'
Another option, __options.intervalMultiplier__, says how many of such intervals are between two time points.  It should be an integer value.  For years and months we truncate them to the desired value, while for days and weeks we will count from a fixed origin day near Epoch (Monday 5 january 1970), so that any day intervals are independent from underlying months and years.  That also means that in case of months the __options.intervalMultiplier__ should be equal to 1, 2, 3, 4 or 6 to be displayed correctly.  To calculate number of days or weeks (reduced to 7 calculation for 7 days) from origin day we use binary subtracting, starting from huge multiplier equal to 1048576 (roughly 2800/11200 years), going down to base multiplier equal to 1.  For interval types smaller or equal than hours we use the __findNextPoint()__ function by incrementing local origin (the interval type and everything smaller is reset to 0).  This is done to avoid problems with daylight-saving and similar things.  In __findNextPoint()__ we make sure that edge points are consistent while using different values of __start__.

      findLeftTime: (start) ->
        leftTime = new Date start.getTime()
        switch @options.intervalType
          when 'year'
            newYear = start.getFullYear() - start.getFullYear() % @options.intervalMultiplier
            leftTime.setFullYear newYear, 0, 1
            leftTime.setHours 0, 0, 0, 0
          when 'month'
            newMonth = start.getMonth() - start.getMonth() % @options.intervalMultiplier
            leftTime.setMonth newMonth, 1
            leftTime.setHours 0, 0, 0, 0
          when 'week', 'day'
            leftTime.setFullYear 1970, 0, 5
            leftTime.setHours 0, 0, 0, 0
            delta = 1048576
            intervalMultiplier = @options.intervalMultiplier
            if @options.intervalType is 'week'
              intervalMultiplier *= 7
            while delta >= 1
              nextTime = new Date leftTime.getFullYear(),
                leftTime.getMonth()
                leftTime.getDate() + delta*intervalMultiplier
                leftTime.getHours()
                leftTime.getMinutes()
                leftTime.getSeconds()
                leftTime.getMilliseconds()
              delta /= 2
              leftTime = nextTime if nextTime <= start
          when 'hour'
            leftTime.setHours 0, 0, 0, 0
            while leftTime < start
              nextTime = @findNextPoint leftTime
              leftTime = nextTime if nextTime <= start
          when 'minute'
            newMinutes = start.getMinutes() - start.getMinutes() % @options.intervalMultiplier
            leftTime.setMinutes newMinutes, 0, 0
          when 'second'
            leftTime.setSeconds start.getSeconds(), 0
            while leftTime < start
              nextTime = @findNextPoint leftTime
              leftTime = nextTime if nextTime <= start
          when 'millisecond'
            newMilliseconds = start.getMilliseconds() - start.getMilliseconds() % @options.intervalMultiplier
            leftTime.setMilliseconds newMilliseconds
        leftTime

There is a code repetition in the switch statement above.  One can think of a good way to improve this part of code.

### The __findNextPoint()__ function

This function calculates next edge time point for a current options assuming __timePoint__ argument is an edge time point.  We take care of daylight-saving time and leap seconds here, assuming that at most one hour/minute/second/millisecond is added or subtracted.  Also we assume that subtraction is not happening at time with 0 value, that is something like 00 -> 23 is not happening.

In current implementation we cope with daylight-saving time by adding hours in UTC to date and then checking how the value changes in local time and fixing by one hour if needed.  The same for leap seconds.

      findNextPoint: (timePoint) ->
        nextTime = new Date timePoint.getTime()
        switch @options.intervalType
          when 'year'
            nextTime.setFullYear (timePoint.getFullYear() + @options.intervalMultiplier)
          when 'month'
            nextTime.setMonth (timePoint.getMonth() + @options.intervalMultiplier)
          when 'week'
            nextTime.setDate (timePoint.getDate() + 7*@options.intervalMultiplier)
          when 'day'
            nextTime.setDate (timePoint.getDate() + @options.intervalMultiplier)
          when 'hour'
            nextTime.setUTCHours (timePoint.getUTCHours() + @options.intervalMultiplier)
            hours = nextTime.getHours()
            if hours % @options.intervalMultiplier isnt 0
              if (hours+1) % @options.intervalMultiplier is 0
                nextTime.setUTCHours (nextTime.getUTCHours() + 1)
              else
                nextTime.setUTCHours (nextTime.getUTCHours() - 1)
          when 'minute'
            nextTime.setMinutes (timePoint.getMinutes() + @options.intervalMultiplier)
          when 'second'
            nextTime.setUTCSeconds (timePoint.getUTCSeconds() + @options.intervalMultiplier)
            seconds = nextTime.getSeconds()
            if seconds % @options.intervalMultiplier isnt 0
              if (seconds+1) % @options.intervalMultiplier is 0
                nextTime.setUTCSeconds (nextTime.getUTCSeconds() + 1)
              else
                nextTime.setUTCSeconds (nextTime.getUTCSeconds() - 1)
          when 'millisecond'
            nextTime.setUTCMilliseconds (timePoint.getUTCMilliseconds() + @options.intervalMultiplier)
        nextTime

#### The __timeToCoord()__ function

To get coordinate of time point we use `@intervalLength` that we stored in `formatTimeAxis()`, which is equal to the number of milliseconds between `end` and `start` of the interval.  We use it to calculate pixe/time ratio, equal to `@width / @intervalLength`.

      timeToCoord: (time) ->
        timeFromStart = time - @start
        coordinate = timeFromStart * @width / @intervalLength

#### The __renderToCanvas()__ function

This function renders formatted time axis to html canvas.
 * `axisData` is formatted data (that we get by calling any of the `format...()` functions)
 * `canvas` is a html canvas object on which axis is drawn
 * `left` and `top` are x- and y-coordinates of the canvas that correspond to the (0, 0) point of the axis viewport

      renderToCanvas: (axisData, canvas, left, top) ->

In the beginning we do just regular initialization of context and its properties.  We also wrap our drawing code into `save()` and `restore()` to not alter context properties globally.

        context = canvas.getContext '2d'
        context.save()
        context.strokeStyle = '#000000'
        context.fillStyle = '#000000'
        context.lineWidth = 2
        context.font = 'normal 12px sans-serif'
        context.textAlign = 'center'
        context.textBaseline = 'top'

To all drawing functions we pass coordinates altered by `roundForCanvas()` function, which rounds values in such a way that the resulting graphics is more sharp (especially horizontal and vertical lines and text).

        for tick in axisData.ticks
          x = left + tick.x
          yTop = top + tick.yTop
          yBottom = top + tick.yBottom
          context.moveTo @roundForCanvas(x), @roundForCanvas(yTop)
          context.lineTo @roundForCanvas(x), @roundForCanvas(yBottom)
        context.stroke()

        for textLabel in axisData.textLabels
          x = left + textLabel.x
          y = top + textLabel.y
          context.fillText textLabel.text, x, y

        context.restore()

#### The `roundForCanvas()` function

This function may be used to avoud aliasing, especially on low-resolution displays.  It rounds coordinate to middle-pixel values, which are .5 in case of html canvas.

      roundForCanvas: (coord) ->
        0.5 + Math.round(coord-0.5)

Library tests
-------------

Examples
--------

This simple code displays time axis when html page is loaded, in `timeline` canvas element.

    makeDemo = ->
      canvas = document.getElementById 'timeline'
      context = canvas.getContext '2d'
      context.fillStyle = '#77FFBB'
      context.fillRect(0, 0, canvas.clientWidth, canvas.clientHeight)
      context.stroke()
      timelineMaker = new TimelineMaker {tickLength: 25, intervalType: 'day', intervalMultiplier: 1}
      start = new Date('2015-06-15T00:00:00')
      end = new Date('2015-07-13T15:23:49')
      axisData = timelineMaker.formatTimeAxis {start, end}, canvas.width
      timelineMaker.renderToCanvas axisData, canvas, 0, 15

    window.onload = makeDemo


Handle createdAt and updatedAt for client models

# Object-relational mapping

*This is mostly stubbed for the browser. I can't justify sending a 400k ORM
library to the client, even though it would be awesome.*

    Promise = require 'bluebird'

    pkgman = require 'pkgman'

    collections = {}

    exports.pkgmanRegister = (registrar) ->

#### Implements hook `shrubAngularService`.

      registrar.registerHook 'shrubAngularService', -> [
        '$http'
        ($http) ->

          service = {}

          exports.initialize()

## orm.collection

* (String) `identity` - Collection identity. e.g. `'shrub-user'`

*Get a collection by identity.*

          service.collection = (identity) -> collections[identity]

## orm.collections

*Get all collections.*

          service.collections = -> collections

          service

      ]

    exports.initialize = ->

#### Invoke hook `shrubOrmCollections`.

      collections_ = {}
      for collectionList in pkgman.invokeFlat 'shrubOrmCollections'
        for identity, collection of collectionList

Collection defaults.

          collection.identity ?= identity
          collections_[collection.identity] = collection

Instantiate a model with defaults supplied.

          collection.instantiate = (values = {}) ->
            model = JSON.parse JSON.stringify values

            if not @autoCreatedAt? or @autoCreatedAt is true
              model.createdAt = new Date values.createdAt ? Date.now()

            if not @autoUpdatedAt? or @autoUpdatedAt is true
              model.updatedAt = new Date values.updatedAt ? Date.now()

            for key, value of @attributes

Set functions.

              model[key] = value if 'function' is typeof value

Set any model defaults.

              if value.defaultsTo?
                model[key] ?= if 'function' is typeof value.defaultsTo
                  value.defaultsTo.call model
                else
                  JSON.parse JSON.stringify value.defaultsTo

Handle dates.

              if model[key]? and 'date' is value or 'date' is value.type
                model[key] = new Date model[key]

            model

#### Invoke hook `shrubOrmCollectionsAlter`.

      pkgman.invoke 'shrubOrmCollectionsAlter', collections_

      collections = collections_

# Object-relational mapping

*This is mostly stubbed for the browser. I can't justify sending a 400k ORM
library to the client, even though it would be awesome.*

    Promise = require 'bluebird'

    pkgman = require 'pkgman'

    collections = {}

    exports.pkgmanRegister = (registrar) ->

#### Implements hook `shrubAngularService`.

      registrar.registerHook 'shrubAngularService', -> [
        '$http'
        ($http) ->

          service = {}

          exports.initialize()

## orm.collection

* (String) `identity` - Collection identity. e.g. `'shrub-user'`

*Get a collection by identity.*

          service.collection = (identity) -> collections[identity]

## orm.collections

*Get all collections.*

          service.collections = -> collections

          service

      ]

    exports.initialize = ->

#### Invoke hook `shrubOrmCollections`.

      collections_ = {}
      for collectionList in pkgman.invokeFlat 'shrubOrmCollections'
        for identity, collection of collectionList

Collection defaults.

          collection.identity ?= identity
          collections_[collection.identity] = collection

Instantiate a model with defaults supplied.

          collection.instantiate = (values = {}) ->
            model = JSON.parse JSON.stringify values

            for key, value of @attributes

Set functions.

              model[key] = value if 'function' is typeof value

Set any model defaults.

              if value.defaultsTo?

                model[key] ?= if 'function' is typeof value.defaultsTo
                  value.defaultsTo.call model
                else
                  JSON.parse JSON.stringify value.defaultsTo

Handle dates.

              if model[key]? and 'date' is value or 'date' is value.type
                model[key] = new Date model[key]

            model

#### Invoke hook `shrubOrmCollectionsAlter`.

      pkgman.invoke 'shrubOrmCollectionsAlter', collections_

      collections = collections_


missed these setupMockedResponses

missed these setupMockedResponses

# Batman Request
 Request is an abstraction around XHR requests.  It can be used with a variety of platforms, and is implemented in the platform layer files.

### Platform Request Implmentation Libraries:
  Depending on the platform library the underlying request implementation will change:

 - `batman.jquery.js`: jQuery XHR
 - `batman.solo.js`: Reqwest.js

 _note_ For the tests there are some mocked responses, in a normal non testing situation you would not use these

### How to use Batman Request

  Batman.Request can be used a variety of ways.  This is the underlying library that the `rest_storage_adapater` uses to communicate.

  In other cases where you want to communicate with a server that isn't your default storage_adapter you would use it like:

    test "Send a simple Batman.Request", ->
      Batman.Request.setupMockedResponse()
      Batman.Request.addMockedResponse( "GET", "http://batmanjs.org", ->{})
      new Batman.Request
        url: "http://batmanjs.org"
        method: "GET"
        success: (result) ->
          ok true

### Lifecycle events
 There are a variety of lifecycle events that get fired in a `Request`.  Each of these are passed in as parameters in the options object in the `Request` constructor.

 1. `loading()`: fired before `send()` has been called.

    test "Demonstrate the loading event", 1, ->
       new Batman.Request
         url: "http:://batmanjs.org"
         loading: ->
           ok true, "After send but before the request is made"

 2. `loaded()`: fired after the request is complete

     test "Demo the loaded event", 1, ->
       Batman.Request.setupMockedResponse()
       Batman.Request.addMockedResponse("GET", "http://batmanjs.org", -> {})
       new Batman.Request
         url: "http://batmanjs.org"
         loaded: ->
           ok true, "Loaded"

 3. `error( xhr )`: fired when an error is detected,  `xhr` is the platform defined xhr object with a property named `request` being the `Batman.Request` object.

    test "Demo the error event", 1, ->
      Batman.Request.setupMockedResponse()
      Batman.Request.addMockedResponse("GET", "http://batmanjs.org", -> status: 400)
      new Batman.Request
        url: "http://batmanjs.org"
        error: (xhr) ->
          ok true, "Error"

 4. `success ( response )`: fired when a successful request has been made, `response` is the body of the request response.

    test "Demonstrate the success event", 1,  ->
      Batman.Request.setupMockedResponse()
      Batman.Request.addMockedResponse("GET", "http://batmanjs.org", -> response: "foo")
      new Batman.Request
        url: "http://batmanjs.org"
        success: (response) ->
          equal "foo", response

## .status
  The response status, only set after `error` or `success`

    test "Demo the error event", 1, ->
     Batman.Request.setupMockedResponse()
     Batman.Request.addMockedResponse("GET", "http://batmanjs.org", -> status: 400)
     new Batman.Request
       url: "http://batmanjs.org"
       error: (xhr) ->
         equal 400, @status

## .response
  The response object

    test "Demonstrate the response object", 1, ->
      Batman.Request.setupMockedResponse()
      Batman.Request.addMockedResponse("GET", "http://batmanjs.org", -> response: "foo" )

      new Batman.Request
        url: "http://batmanjs.org"
        success: (response) ->
          equal "foo", JSON.parse(@response)

## .responseHeaders : object
  An key/value object with the response headers

    test "responseHeaders", 1, ->
      Batman.Request.setupMockedResponse()
      Batman.Request.addMockedResponse("GET", "http://batmanjs.org", -> responseHeaders: { FOO: "bar" } )

      new Batman.Request
        url: "http://batmanjs.org"
        success: (response) ->
          deepEqual @responseHeaders,{ FOO: "bar" }

## @Request(options : object)

  Constructs a new `Batman.Request` with the supplied options

  `options` description
  1. `url : String`: A string containing the URL to which the request is sent.  Mandatory
  2. `method : String`: The type of request to make eg. `GET`, `POST`.  Default is `GET`

    test "method test", ->
      Batman.Request.setupMockedResponse()
      Batman.Request.addMockedResponse("POST", "http://batmanjs.org", -> response : { FOO: "BAR" })
      new Batman.Request
        url: "http://batmanjs.org"
        method: "POST"
        data: { some: "data" }
        success: (response) ->
          ok true

  3. `data`: Data to be sent to the server. It is converted to a query string, if not already a string. It's appended to the url for GET-requests.
  4. `contentType`: the content type to send, defaults to `application/x-www-form-urlencoded`

    test "contentType default", ->
       req = new Batman.Request
       equal req.contentType, "application/x-www-form-urlencoded"

  5. `autosend`: controls whether the request will be sent automatically, deafults to `true`

    test "autosend test", ->
      response_sent = false
      Batman.Request.setupMockedResponse()
      Batman.Request.addMockedResponse("GET", "http://batmanjs.org", -> {} )
      req = new Batman.Request
        url: "http://batmanjs.org"
        autosend: false
        success: (response) ->
          response_sent = true
      ok !response_sent
      req.send()
      ok response_sent

  6. `username`: username for authentication
  7. `password`: password for authentication
  8. `type`: The type of data you're expecting from the server, defaults to `json`

## ::send([data])
  If `autosend` is `false`, this will initiate the request.  The data object passed in will take priority over the one specified in the `options` object

    test "request.send()", ->
      beforeResponse = (req, data) ->
        deepEqual data, { other: "thing" }
      Batman.Request.setupMockedResponse()
      Batman.Request.addMockedResponse("GET", "http://batmanjs.org", -> {beforeResponse: beforeResponse} )
      req = new Batman.Request
        url: "http://batmanjs.org"
        autosend: false
        data: { some: "data" }
      req.send( { other: "thing" } )

# Batman Request
 Request is an abstraction around XHR requests.  It can be used with a variety of platforms, and is implemented in the platform layer files.

### Platform Request Implmentation Libraries:
  Depending on the platform library the underlying request implementation will change:

 - `batman.jquery.js`: jQuery XHR
 - `batman.solo.js`: Reqwest.js

 _note_ For the tests there are some mocked responses, in a normal non testing situation you would not use these

### How to use Batman Request

  Batman.Request can be used a variety of ways.  This is the underlying library that the `rest_storage_adapater` uses to communicate.

  In other cases where you want to communicate with a server that isn't your default storage_adapter you would use it like:

    test "Send a simple Batman.Request", ->
      Batman.Request.setupMockedResponse()
      Batman.Request.addMockedResponse( "GET", "http://batmanjs.org", ->{})
      new Batman.Request
        url: "http://batmanjs.org"
        method: "GET"
        success: (result) ->
          ok true

### Lifecycle events
 There are a variety of lifecycle events that get fired in a `Request`.  Each of these are passed in as parameters in the options object in the `Request` constructor.

 1. `loading()`: fired before `send()` has been called.

    test "Demonstrate the loading event", 1, ->
       new Batman.Request
         url: "http:://batmanjs.org"
         loading: ->
           ok true, "After send but before the request is made"

 2. `loaded()`: fired after the request is complete

     test "Demo the loaded event", 1, ->
       Batman.Request.setupMockedResponse()
       Batman.Request.addMockedResponse("GET", "http://batmanjs.org", -> {})
       new Batman.Request
         url: "http://batmanjs.org"
         loaded: ->
           ok true, "Loaded"

 3. `error( xhr )`: fired when an error is detected,  `xhr` is the platform defined xhr object with a property named `request` being the `Batman.Request` object.

    test "Demo the error event", 1, ->
      Batman.Request.setupMockedResponse()
      Batman.Request.addMockedResponse("GET", "http://batmanjs.org", -> status: 400)
      new Batman.Request
        url: "http://batmanjs.org"
        error: (xhr) ->
          ok true, "Error"

 4. `success ( response )`: fired when a successful request has been made, `response` is the body of the request response.

    test "Demonstrate the success event", 1,  ->
      Batman.Request.setupMockedResponse()
      Batman.Request.addMockedResponse("GET", "http://batmanjs.org", -> response: "foo")
      new Batman.Request
        url: "http://batmanjs.org"
        success: (response) ->
          equal "foo", response

## .status
  The response status, only set after `error` or `success`

    test "Demo the error event", 1, ->
     Batman.Request.setupMockedResponse()
     Batman.Request.addMockedResponse("GET", "http://batmanjs.org", -> status: 400)
     new Batman.Request
       url: "http://batmanjs.org"
       error: (xhr) ->
         equal 400, @status

## .response
  The response object

    test "Demonstrate the response object", 1, ->
      Batman.Request.setupMockedResponse()
      Batman.Request.addMockedResponse("GET", "http://batmanjs.org", -> response: "foo" )

      new Batman.Request
        url: "http://batmanjs.org"
        success: (response) ->
          equal "foo", JSON.parse(@response)

## .responseHeaders : object
  An key/value object with the response headers

    test "responseHeaders", 1, ->
      Batman.Request.setupMockedResponse()
      Batman.Request.addMockedResponse("GET", "http://batmanjs.org", -> responseHeaders: { FOO: "bar" } )

      new Batman.Request
        url: "http://batmanjs.org"
        success: (response) ->
          deepEqual @responseHeaders,{ FOO: "bar" }

## @Request(options : object)

  Constructs a new `Batman.Request` with the supplied options

  `options` description
  1. `url : String`: A string containing the URL to which the request is sent.  Mandatory
  2. `method : String`: The type of request to make eg. `GET`, `POST`.  Default is `GET`

    test "method test", ->
      Batman.Request.addMockedResponse("POST", "http://batmanjs.org", -> response : { FOO: "BAR" })
      new Batman.Request
        url: "http://batmanjs.org"
        method: "POST"
        data: { some: "data" }
        success: (response) ->
          ok true

  3. `data`: Data to be sent to the server. It is converted to a query string, if not already a string. It's appended to the url for GET-requests.
  4. `contentType`: the content type to send, defaults to `application/x-www-form-urlencoded`

    test "contentType default", ->
       req = new Batman.Request
       equal req.contentType, "application/x-www-form-urlencoded"

  5. `autosend`: controls whether the request will be sent automatically, deafults to `true`

    test "autosend test", ->
      response_sent = false
      Batman.Request.addMockedResponse("GET", "http://batmanjs.org", -> {} )
      req = new Batman.Request
        url: "http://batmanjs.org"
        autosend: false
        success: (response) ->
          response_sent = true
      ok !response_sent
      req.send()
      ok response_sent

  6. `username`: username for authentication
  7. `password`: password for authentication
  8. `type`: The type of data you're expecting from the server, defaults to `json`

## ::send([data])
  If `autosend` is `false`, this will initiate the request.  The data object passed in will take priority over the one specified in the `options` object

    test "request.send()", ->
      beforeResponse = (req, data) ->
        deepEqual data, { other: "thing" }
      Batman.Request.setupMockedResponse()
      Batman.Request.addMockedResponse("GET", "http://batmanjs.org", -> {beforeResponse: beforeResponse} )
      req = new Batman.Request
        url: "http://batmanjs.org"
        autosend: false
        data: { some: "data" }
      req.send( { other: "thing" } )


Dead code

Dead code

# HTTP

Manage HTTP connections.

    config = require 'config'
    pkgman = require 'pkgman'

    debug = require('debug') 'shrub:http'

    httpManager = null

    exports.pkgmanRegister = (registrar) ->

#### Implements hook `shrubCoreBootstrapMiddleware`.

      registrar.registerHook 'shrubCoreBootstrapMiddleware', ->

        label: 'Bootstrap HTTP server'
        middleware: [

          (next) ->

            {manager, port} = config.get 'packageSettings:shrub-http'

            {Manager} = require manager.module

Spin up the HTTP server, and initialize it.

            httpManager = new Manager()

Spawn workers into a cluster.

            httpManager.cluster()

            httpManager.initialize().then(->

              debug "Shrub HTTP server up and running on port #{port}!"
              next()

            ).catch next

        ]

#### Implements hook `shrubHttpMiddleware`.

      registrar.registerHook 'shrubHttpMiddleware', (http) ->

        label: 'Finalize HTTP request'
        middleware: [

          (req, res, next) ->

            if req.delivery?

              res.send req.delivery

            else

              res.status 501
              res.end '
501 Internal Server Error
'

        ]

#### Implements hook `shrubConfigServer`.

      registrar.registerHook 'shrubConfigServer', ->

        manager:

Module implementing the socket manager.

          module: 'shrub-http-express'

        middleware: [
          'shrub-http-express/static'
          'shrub-core'
          'shrub-socket/factory'
          'shrub-http-express/session'
          'shrub-user'
          'shrub-http-express/logger'
          'shrub-villiany'
          'shrub-form'
          'shrub-http-express/routes'
          'shrub-config'
          'shrub-skin/path'
          'shrub-assets'
          'shrub-skin/render'
          'shrub-http-express/errors'
        ]

        path: "#{config.get 'path'}/app"

        port: 4201

    exports.manager = -> httpManager

# HTTP

Manage HTTP connections.

    config = require 'config'
    pkgman = require 'pkgman'

    debug = require('debug') 'shrub:http'

    httpManager = null

    exports.pkgmanRegister = (registrar) ->

#### Implements hook `shrubCoreBootstrapMiddleware`.

      registrar.registerHook 'shrubCoreBootstrapMiddleware', ->

        label: 'Bootstrap HTTP server'
        middleware: [

          (next) ->

            {manager, port} = config.get 'packageSettings:shrub-http'

            {Manager} = require manager.module

Spin up the HTTP server, and initialize it.

            httpManager = new Manager()

Spawn workers into a cluster.

            httpManager.cluster()

            httpManager.initialize().then(->

              debug "Shrub HTTP server up and running on port #{port}!"
              next()

            ).catch next

        ]

#### Implements hook `shrubHttpMiddleware`.

      registrar.registerHook 'shrubHttpMiddleware', (http) ->

        label: 'Finalize HTTP request'
        middleware: [

          (req, res, next) ->

            if req.delivery?

              res.send req.delivery

            else

              res.status 501
              res.end '
501 Internal Server Error
'

        ]

#### Implements hook `shrubConfigServer`.

      registrar.registerHook 'shrubConfigServer', ->

        manager:

Module implementing the socket manager.

          module: 'shrub-http-express'

        middleware: [
          'shrub-http-express/static'
          'shrub-core'
          'shrub-socket/factory'
          'shrub-http-express/session'
          'shrub-user'
          'shrub-http-express/logger'
          'shrub-villiany'
          'shrub-form'
          'shrub-http-express/routes'
          'shrub-config'
          'shrub-skin/path'
          'shrub-assets'
          'shrub-skin/render'
          'shrub-angular'
          'shrub-http-express/errors'
        ]

        path: "#{config.get 'path'}/app"

        port: 4201

    exports.manager = -> httpManager


Dox look a little nicer now

Dox look a little nicer now

# Grunt build process - Dynamic documentation

    {exec} = require 'child_process'
    fs = require 'fs'
    path = require 'path'
    {Transform} = require 'stream'

    _ = require 'lodash'
    {LineStream} = require 'byline'
    glob = require 'grunt/node_modules/glob'
    Promise = require 'bluebird'

Implement a Transform stream to accumulate hook invocations from a source file.

    class HookInvocations extends Transform

      constructor: ->
        super

        @list = []

      _transform: (chunk, encoding, done) ->
        line = chunk.toString('utf8')
        if matches = line.match /^\#\#\#\# Invoke hook `([^`]+)`/
          @list.push matches[1]

        done()

Implement a Transform stream to accumulate hook implementations from a source
file.

    class HookImplementations extends Transform

      constructor: ->
        super

        @list = []

      _transform: (chunk, encoding, done) ->
        line = chunk.toString('utf8')
        if matches = line.match /^\#\#\#\# Implements hook `([^`]+)`/
          @list.push matches[1]

        done()

Implement a Transform stream fo accumulate TODOs from a source file. Also
caches lines to be able to build context around each TODO item.

    class Todos extends Transform

      @context = 4

      constructor: ->
        super

        @lines = []
        @todos = []

      _transform: (chunk, encoding, done) ->
        @lines.push line = chunk.toString('utf8')
        if matches = line.match /^\#\#\#\#\#\# TODO/
          @todos.push @lines.length - 1

        done()

      withContext: ->

        for todo in @todos

          start = Math.max 0, todo - Todos.context
          end = Math.min @lines.length - 1, todo + Todos.context

          index: todo
          lines: @lines.slice start, end

Gather all source files.

    _allSourceFiles = ->
      new Promise (resolve, reject) ->
        glob(
          '{{client,custom,packages,server}/**/*.litcoffee,*.litcoffee}'
          (error, files) ->
            return reject error if error?
            resolve files
        )

Generate an HTML ID from a string.

    _idFromString = (string) -> string.replace(
      /\//g, ''
    ).replace(
      /[^0-9A-Za-z-]+/g, '-'
    ).replace(
      /\-+/g, '-'
    ).toLowerCase()

Get the source path from a filename. This removes the extension and any /index
portion from the end of the filename.

    _sourcePath = (filename) ->

        dirname = path.dirname filename
        if dirname is '.' then dirname = '' else dirname += '/'
        extname = path.extname filename
        filename = "#{dirname}#{path.basename filename, extname}"

        parts = filename.split '/'
        parts.pop() if parts[parts.length - 1] is 'index'
        return parts.join '/'

Collate a list of files by type (client or server).

    _collateFilesByType = (files) ->

      client = []
      server = []

      for file in files
        parts = file.split '/'

        if parts[0] is 'client'
          client.push file
        else if parts[0] is 'custom' and parts[2] is 'client'
          client.push file
        else if parts[0] is 'packages' and parts[2] is 'client'
          client.push file
        else
          server.push file

      client: client, server: server

Gather statistics for all files.

    fileStatsListPromise = _allSourceFiles().then (allFiles) ->

      allFilesPromises = for type, files of _collateFilesByType allFiles

        typePromises = for file in files

          do (type, file) -> new Promise (resolve, reject) ->

            fstream = fs.createReadStream file
            fstream.pipe lineStream = new LineStream keepEmptyLines: true

Pass all files through the Transform list to parse out relevant information.

            lineStream.pipe hookImplementations = new HookImplementations()
            lineStream.pipe hookInvocations = new HookInvocations()
            lineStream.pipe todos = new Todos()

            fstream.on 'error', reject

            fstream.on 'end', ->

Include all information from Transform streams in the statistics.

              resolve(
                type: type
                file: file
                implementations: hookImplementations.list
                invocations: hookInvocations.list
                todos: todos.withContext()
              )

        Promise.all typePromises

      Promise.all(allFilesPromises).then (fileStatsLists) ->
        _.flatten fileStatsLists

Massage the statistics to help rendering the hooks page.

    fileStatsListPromise.then((fileStatsList) ->

      hooksIndex = {}

      indexes = {}
      keys = ['implementations', 'invocations']

      for fileStats in fileStatsList

        parts = fileStats.file.split '/'
        for remove in ['client', 'packages']
          if ~(index = parts.indexOf remove)
            parts.splice index, 1
        mergeFile = parts.join '/'

        for key in keys
          indexes[key] ?= {}

          for hook in fileStats[key]
            indexes[key][hook] ?= {}

            indexes[key][hook][mergeFile] ?= fullName: fileStats.file
            (indexes[key][hook][mergeFile].types ?= []).push fileStats.type

            hooksIndex[hook] = true

      for key in keys
        for hook, stats of indexes[key]
          for file of stats
            stats[file].types = _.unique stats[file].types ? []

      hooks = (hook for hook of hooksIndex).sort()

      hookFiles = {}
      for hook in hooks
        hookFiles[hook] = try
          fs.readFileSync "docs/hook/#{hook}.md", 'utf8'
        catch error
          ''

      O =
        hookFiles: hookFiles
        hooks: hooks

      O[key] = indexes[key] for key in keys

      return O

Render the hooks page.

    ).then((O) ->

      {hookFiles, hooks} = O

      keys = ['implementation', 'invocation']

      wordingFor =
        implementation: 'implements'
        invocation: 'invoke'

      render = fs.readFileSync 'docs/hooks.template.md', 'utf8'

      for hook in hooks

        render += "## #{hook}\n\n"
        render += hookFiles[hook] + '\n\n' if hookFiles[hook]

        for key in keys
          pluralKey = "#{key}s"

          if O[pluralKey][hook]?

            count = Object.keys(O[pluralKey][hook]).length

            render += "!!! note \"#{count} #{key}"
            render += 's' if count > 1
            render += '"\n'

            instances = for file, {fullName, types} of O[pluralKey][hook]

Remove packages and file part.

              parts = fullName.split '/'
              parts.shift()
              parts.pop()
              packageName = parts.join '/'

              types = types.map (type) ->
                packageNameParts = packageName.split '/'
                packageNameParts.pop() if packageNameParts[packageNameParts.length - 1] isnt type
                "    * [#{_sourcePath file} (#{type})](packages/##{_idFromString packageNameParts.join '/'}) — [#{key}](source/#{_sourcePath fullName}##{wordingFor[key]}-hook-#{_idFromString hook}) \n"
              types.join ''

            render += instances.join ''

            render += '\n\n'

      new Promise (resolve, reject) ->
        fs.writeFile 'docs/hooks.md', render, (error) ->
          return reject error if error?
          resolve()

    ).done()

Render the TODOs page.

    fileStatsListPromise.then((fileStatsList) ->

      render = fs.readFileSync 'docs/todos.template.md', 'utf8'

      for fileStats in fileStatsList

Keep track of used IDs, it will be necessary to link to the correct location
hash in the case of multiple TODO items with the same wording.

        idMap = {}

        for todo in fileStats.todos

          render += '\n'

          id = ''
          for line, index in todo.lines
            render += '>'

If this is the line with the TODO, parse the ID from the TODO item text, and
render it as h2 (TODO are h6) to increase visibility.

            if index is Todos.context
              id = _idFromString(line).slice 1, -1

              render += line.slice 4
            else
              render += " #{line}"
            render += '\n'

          render += '\n'

          filename = _sourcePath fileStats.file

Keep track of ID usage and modify the location hash for subsequent uses.

          if idMap[id]?
            idMap[id] += 1
            id += "_#{idMap[id]}"
          else
            idMap[id] = 0

          render += "###### the above found in [#{fileStats.file}:#{todo.index}](source/#{filename}##{id})\n"

      new Promise (resolve, reject) ->
        fs.writeFile 'docs/todos.md', render, (error) ->
          return reject error if error?
          resolve()

    ).done()

Add all the source files to a generated mkdocs.yml

    _allSourceFiles().then (files) ->

      yml = fs.readFileSync 'docs/mkdocs.template.yml', 'utf8'

      for file in files
        parts = file.split '/'

        for i in [0...parts.length]
          try
            fs.mkdirSync "docs/source/#{parts.slice(0, i).join '/'}"
          catch error

        fs.createReadStream(file).pipe(
          fs.createWriteStream("docs/source/#{file}")
        )

Add them under the 'Source code' path.

        yml += "- [source/#{file}, 'Source code', '#{file}']\n"

      fs.writeFileSync 'mkdocs.yml', yml

      return

Render the packages page.

    fileStatsListPromise.then((fileStatsList) ->

Sort by package name first.

      newList = []

      for fileStats in fileStatsList

        parts = fileStats.file.split '/'
        continue unless ~['custom', 'packages'].indexOf parts[0]

        sourcePath = _sourcePath fileStats.file

        parts = sourcePath.split '/'
        # parts.pop() if parts[parts.length - 1] is 'client'
        pkg = parts.join '/'
        fileStats.pkg = pkg.split('/').slice(1).join '/'

        newList.push fileStats

      newList.sort (l, r) ->
        return -1 if l.type is 'client' and r.type is 'server'
        return 1 if l.type is 'server' and r.type is 'client'
        if l.pkg < r.pkg then -1 else if l.pkg > r.pkg then 1 else 0

    ).then((fileStatsList) ->

      render = fs.readFileSync 'docs/packages.template.md', 'utf8'
      render += '\n'

      type = null

      for fileStats in fileStatsList

        if fileStats.type isnt type
          type = fileStats.type

          render += if type is 'client'
            '## Client-side'
          else
            '## Server-side'

          render += '\n\n'

        pkgParts = fileStats.pkg.split '/'
        isSubpackage = pkgParts.length isnt 1 and pkgParts.pop() isnt 'client'

Link to the package.

        sourcePath = _sourcePath fileStats.file

        if isSubpackage
          render += '> '
          parts = fileStats.pkg.split '/'
          parentPkg = parts.shift()
          subPkg = parts.join '/'
          render += "## [#{parentPkg}/#{subPkg}](source/#{sourcePath})"
        else
          render += "## [#{fileStats.pkg}](source/#{sourcePath})"

Naively parse out the file description. It must be wrapped in asterisks, i.e.
italicized in markdown.

###### TODO: This 'chunk' parsing should be done with a Transform like the others.

        data = fs.readFileSync fileStats.file, 'utf8'
        chunks = data.split '\n\n'

        title = chunks[0]
        if 35 is title.charCodeAt 0
          render += '\n\n'
          render += '> ' if isSubpackage
          render += "#{title.substr 2}"

        render += '\n\n'

        description = chunks[1]
        if 42 is description.charCodeAt 0 and 42 is description.charCodeAt description.length - 1
          render += '> ' if isSubpackage
          render += "#{description}\n\n"

        if fileStats.implementations.length > 0
          render += '> ' if isSubpackage
          render += '!!! note "Implements hooks"\n'
          render += fileStats.implementations.map((hook) ->
            "    * [#{hook}](hooks/##{_idFromString hook}) — [implementation](source/#{sourcePath}#implements-hook-#{hook.toLowerCase()})\n"
          ).join ''
          render += '\n\n'

        if fileStats.invocations.length > 0
          render += '> ' if isSubpackage
          render += '!!! note "Invokes hooks"\n'
          render += fileStats.invocations.map((hook) ->
            "    * [#{hook}](hooks/##{_idFromString hook}) — [invocation](source/#{sourcePath}#invoke-hook-#{hook.toLowerCase()})\n"
          ).join ''
          render += '\n\n'

###### TODO: We should do some preprocessing hre with a transform, namely linking the hook headers to their respective documentation.

      fs.writeFileSync 'docs/packages.md', render

    )

# Grunt build process - Dynamic documentation

    {exec} = require 'child_process'
    fs = require 'fs'
    path = require 'path'
    {Transform} = require 'stream'

    _ = require 'lodash'
    {LineStream} = require 'byline'
    glob = require 'grunt/node_modules/glob'
    Promise = require 'bluebird'

Implement a Transform stream to accumulate hook invocations from a source file.

    class HookInvocations extends Transform

      constructor: ->
        super

        @list = []

      _transform: (chunk, encoding, done) ->
        line = chunk.toString('utf8')
        if matches = line.match /^\#\#\#\# Invoke hook `([^`]+)`/
          @list.push matches[1]

        done()

Implement a Transform stream to accumulate hook implementations from a source
file.

    class HookImplementations extends Transform

      constructor: ->
        super

        @list = []

      _transform: (chunk, encoding, done) ->
        line = chunk.toString('utf8')
        if matches = line.match /^\#\#\#\# Implements hook `([^`]+)`/
          @list.push matches[1]

        done()

Implement a Transform stream fo accumulate TODOs from a source file. Also
caches lines to be able to build context around each TODO item.

    class Todos extends Transform

      @context = 4

      constructor: ->
        super

        @lines = []
        @todos = []

      _transform: (chunk, encoding, done) ->
        @lines.push line = chunk.toString('utf8')
        if matches = line.match /^\#\#\#\#\#\# TODO/
          @todos.push @lines.length - 1

        done()

      withContext: ->

        for todo in @todos

          start = Math.max 0, todo - Todos.context
          end = Math.min @lines.length - 1, todo + Todos.context

          index: todo
          lines: @lines.slice start, end

Gather all source files.

    _allSourceFiles = ->
      new Promise (resolve, reject) ->
        glob(
          '{{client,custom,packages,server}/**/*.litcoffee,*.litcoffee}'
          (error, files) ->
            return reject error if error?
            resolve files
        )

Generate an HTML ID from a hook name.

    _idFromHook = (hook) -> hook.replace(
      /[^0-9A-Za-z-]+/g, '-'
    ).replace(
      /\-+/g, '-'
    ).toLowerCase()

Get the source path from a filename. This removes the extension and any /index
portion from the end of the filename.

    _sourcePath = (filename) ->

        dirname = path.dirname filename
        if dirname is '.' then dirname = '' else dirname += '/'
        extname = path.extname filename
        filename = "#{dirname}#{path.basename filename, extname}"

        parts = filename.split '/'
        parts.pop() if parts[parts.length - 1] is 'index'
        return parts.join '/'

Collate a list of files by type (client or server).

    _collateFilesByType = (files) ->

      client = []
      server = []

      for file in files
        parts = file.split '/'

        if parts[0] is 'client'
          client.push file
        else if parts[0] is 'custom' and parts[2] is 'client'
          client.push file
        else if parts[0] is 'packages' and parts[2] is 'client'
          client.push file
        else
          server.push file

      client: client, server: server

Gather statistics for all files.

    fileStatsListPromise = _allSourceFiles().then (allFiles) ->

      allFilesPromises = for type, files of _collateFilesByType allFiles

        typePromises = for file in files

          do (type, file) -> new Promise (resolve, reject) ->

            fstream = fs.createReadStream file
            fstream.pipe lineStream = new LineStream keepEmptyLines: true

Pass all files through the Transform list to parse out relevant information.

            lineStream.pipe hookImplementations = new HookImplementations()
            lineStream.pipe hookInvocations = new HookInvocations()
            lineStream.pipe todos = new Todos()

            fstream.on 'error', reject

            fstream.on 'end', ->

Include all information from Transform streams in the statistics.

              resolve(
                type: type
                file: file
                implementations: hookImplementations.list
                invocations: hookInvocations.list
                todos: todos.withContext()
              )

        Promise.all typePromises

      Promise.all(allFilesPromises).then (fileStatsLists) ->
        _.flatten fileStatsLists

Massage the statistics to help rendering the hooks page.

    fileStatsListPromise.then((fileStatsList) ->

      hooksIndex = {}

      indexes = {}
      keys = ['implementations', 'invocations']

      for fileStats in fileStatsList

        parts = fileStats.file.split '/'
        for remove in ['client', 'packages']
          if ~(index = parts.indexOf remove)
            parts.splice index, 1
        mergeFile = parts.join '/'

        for key in keys
          indexes[key] ?= {}

          for hook in fileStats[key]
            indexes[key][hook] ?= {}

            indexes[key][hook][mergeFile] ?= fullName: fileStats.file
            (indexes[key][hook][mergeFile].types ?= []).push fileStats.type

            hooksIndex[hook] = true

      for key in keys
        for hook, stats of indexes[key]
          for file of stats
            stats[file].types = _.unique stats[file].types ? []

      hooks = (hook for hook of hooksIndex).sort()

      hookFiles = {}
      for hook in hooks
        hookFiles[hook] = try
          fs.readFileSync "docs/hook/#{hook}.md", 'utf8'
        catch error
          ''

      O =
        hookFiles: hookFiles
        hooks: hooks

      O[key] = indexes[key] for key in keys

      return O

Render the hooks page.

    ).then((O) ->

      {hookFiles, hooks} = O

      keys = ['implementation', 'invocation']

      wordingFor =
        implementation: 'implements'
        invocation: 'invoke'

      render = fs.readFileSync 'docs/hooks.template.md', 'utf8'

      for hook in hooks

        render += "## #{hook}\n\n"
        render += hookFiles[hook] + '\n\n' if hookFiles[hook]

        for key in keys
          pluralKey = "#{key}s"

          if O[pluralKey][hook]?

            count = Object.keys(O[pluralKey][hook]).length

            render += "!!! note \"#{count} #{key}"
            render += 's' if count > 1
            render += '"\n'

            instances = for file, {fullName, types} of O[pluralKey][hook]
              types = types.map (type) -> "[#{type}](source/#{_sourcePath fullName}##{wordingFor[key]}-hook-#{_idFromHook hook})"
              "    * #{_sourcePath file} (#{types.join ', '})\n"

            render += instances.join('')

            render += '\n\n'

      new Promise (resolve, reject) ->
        fs.writeFile 'docs/hooks.md', render, (error) ->
          return reject error if error?
          resolve()

    ).done()

Render the TODOs page.

    fileStatsListPromise.then((fileStatsList) ->

      render = fs.readFileSync 'docs/todos.template.md', 'utf8'

      for fileStats in fileStatsList

Keep track of used IDs, it will be necessary to link to the correct location
hash in the case of multiple TODO items with the same wording.

        idMap = {}

        for todo in fileStats.todos

          render += '\n'

          id = ''
          for line, index in todo.lines
            render += '>'

If this is the line with the TODO, parse the ID from the TODO item text, and
render it as h2 (TODO are h6) to increase visibility.

            if index is Todos.context
              id = _idFromHook(line).slice 1, -1

              render += line.slice 4
            else
              render += " #{line}"
            render += '\n'

          render += '\n'

          filename = _sourcePath fileStats.file

Keep track of ID usage and modify the location hash for subsequent uses.

          if idMap[id]?
            idMap[id] += 1
            id += "_#{idMap[id]}"
          else
            idMap[id] = 0

          render += "###### the above found in [#{fileStats.file}:#{todo.index}](source/#{filename}##{id})\n"

      new Promise (resolve, reject) ->
        fs.writeFile 'docs/todos.md', render, (error) ->
          return reject error if error?
          resolve()

    ).done()

Add all the source files to a generated mkdocs.yml

    _allSourceFiles().then (files) ->

      yml = fs.readFileSync 'docs/mkdocs.template.yml', 'utf8'

      for file in files
        parts = file.split '/'

        for i in [0...parts.length]
          try
            fs.mkdirSync "docs/source/#{parts.slice(0, i).join '/'}"
          catch error

        fs.createReadStream(file).pipe(
          fs.createWriteStream("docs/source/#{file}")
        )

Add them under the 'Source code' path.

        yml += "- [source/#{file}, 'Source code', '#{file}']\n"

      fs.writeFileSync 'mkdocs.yml', yml

      return

Render the packages page.

    fileStatsListPromise.then((fileStatsList) ->

Sort by package name first.

      newList = []

      for fileStats in fileStatsList

        parts = fileStats.file.split '/'
        continue unless ~['custom', 'packages'].indexOf parts[0]

        sourcePath = _sourcePath fileStats.file

        parts = sourcePath.split '/'
        parts.pop() if parts[parts.length - 1] is 'client'
        pkg = parts.join '/'
        fileStats.pkg = pkg.split('/').slice(1).join '/'

        newList.push fileStats

      newList.sort (l, r) ->
        return -1 if l.type is 'client' and r.type is 'server'
        return 1 if l.type is 'server' and r.type is 'client'
        if l.pkg < r.pkg then -1 else if l.pkg > r.pkg then 1 else 0

    ).then((fileStatsList) ->

      render = fs.readFileSync 'docs/packages.template.md', 'utf8'
      render += '\n'

      type = null

      for fileStats in fileStatsList

        if fileStats.type isnt type
          type = fileStats.type

          render += if type is 'client'
            '## Client-side'
          else
            '## Server-side'

          render += '\n\n'

        isSubpackage = fileStats.pkg.split('/').length isnt 1

Link to the package.

        sourcePath = _sourcePath fileStats.file

        if isSubpackage
          render += '> '
          parts = fileStats.pkg.split '/'
          parentPkg = parts.shift()
          subPkg = parts.join '/'
          render += "## [#{parentPkg}/#{subPkg}](source/#{sourcePath})"
        else
          render += "## [#{fileStats.pkg}](source/#{sourcePath})"

Naively parse out the file description. It must be wrapped in asterisks, i.e.
italicized in markdown.

###### TODO: This 'chunk' parsing should be done with a Transform like the others.

        data = fs.readFileSync fileStats.file, 'utf8'
        chunks = data.split '\n\n'

        title = chunks[0]
        if 35 is title.charCodeAt 0
          render += '\n\n'
          render += '> ' if isSubpackage
          render += "#{title.substr 2}"

        render += '\n\n'

        description = chunks[1]
        if 42 is description.charCodeAt 0 and 42 is description.charCodeAt description.length - 1
          render += '> ' if isSubpackage
          render += "#{description}\n\n"

        if fileStats.implementations.length > 0
          render += '> ' if isSubpackage
          render += '!!! note "Implements hooks"\n    '
          render += fileStats.implementations.map((hook) ->
            "[#{hook}](source/#{sourcePath}#implements-hook-#{hook.toLowerCase()})"
          ).join ', '
          render += '\n\n'

        if fileStats.invocations.length > 0
          render += '> ' if isSubpackage
          render += '!!! note "Invokes hooks"\n    '
          render += fileStats.invocations.map((hook) ->
            "[#{hook}](source/#{sourcePath}#invoke-hook-#{hook.toLowerCase()})"
          ).join ', '
          render += '\n\n'

###### TODO: We should do some preprocessing hre with a transform, namely linking the hook headers to their respective documentation.

      fs.writeFileSync 'docs/packages.md', render

    )


Rework Skyll's rendering pipeline

Rework Skyll's rendering pipeline

The rendering pipeline now supports nested arrays - each nested array
specifying a complete rendering pipeline.

Data cascades from pipeline to pipeline since pipelines are rendered in
order.