How to normalise unique dates for graphing
Question: I've currently got a data-set from an application, but the problem is the data
is listed whenever it changes, not on a regular interval. For example I could
have 20 entries in one day and then nothing for two days, then one, then one
next month.

I want to graph the data I have, but with a consistent time scale - currently
it just graphs every entry with every date one after the other.

I have thought about the problem and an idea that came to mind would be
determining the start and end date of the data, dividing it by the graphing
resolution desired, (eg 100 entries), then for each entry working out the
closest result in the data-set BEFORE the resolution sample. ie one of the
samples for graphing is 10th of january, it finds the first sample before the
10th of january and uses it. Then with 100 samples you can graph with a
consistent time scale.

Any ideas on how to actually achieve this would be wonderful! I'm looking to
normalize in python and then render in a browser with google chart api.

Here is an example of the data:

    
    
    10/03/2015 10:55    7385498415
    10/03/2015 9:15     7379639094
    10/03/2015 8:55     7376777516
    10/03/2015 8:35     7368304217
    10/03/2015 8:12     7358015859
    8/03/2015 9:22      7358015859
    8/03/2015 4:27      7354221274
    8/03/2015 4:07      7346810719
    7/03/2015 4:25      7339695326
    7/03/2015 4:25      7339698276
    6/03/2015 13:08     7339701226
    6/03/2015 13:04     7317905872
    6/03/2015 12:44     7309771372
    6/03/2015 12:24     7300851599
    6/03/2015 12:04     7292557469
    6/03/2015 11:39     7283439433
    6/03/2015 11:00     7278056128
    6/03/2015 10:41     7267320628
    6/03/2015 10:35     7265158228
    6/03/2015 10:21     7265158228
    6/03/2015 10:01     7255260402
    6/03/2015 7:06      7246047762
    5/03/2015 15:39     7245760885
    5/03/2015 13:41     7170760885
    4/03/2015 21:21     5595760885
    4/03/2015 18:59     5590496476
    4/03/2015 18:59     5585251201
    


Answer: There’s no need to “normalise” the data, as the Google Charts API has support
for dates directly. Not only is adjusting the data more difficult, but it will
also give a skewed impression.

We can start with the [Dates and
Times](https://developers.google.com/chart/interactive/docs/datesandtimes)
section of their documentation:

> To create a new `Date` object, you call the `Date()` constructor with the
> new keyword, with arguments to specify components of the date. These
> arguments take the form of several numbers corresponding to the different
> properties of your date.
>  
>  
>     new Date(Year, Month, Day, Hours, Minutes, Seconds, Milliseconds)
>  

A row in a Google Chart is an `[x, y]` list. The first entry will be one of
these Date() constructors, and the second will be the value. So we need to
take the data you’ve provided, parse the dates and turn them into Date()
constructors. (Note that we can omit the second and millisecond arguments.)

Here’s a simple script that does this, and then wraps each time in a row that
we can supply to a Google Chart:

    
    
    from datetime import datetime
    
    # This string is a copy of the example data set. The newlines have been
    # replaced with '\n' escape codes to save vertical space on Stack Overflow.
    DATASET = """10/03/2015 10:55    7385498415\n10/03/2015 9:15     7379639094\n10/03/2015 8:55     7376777516\n10/03/2015 8:35     7368304217\n10/03/2015 8:12     7358015859\n8/03/2015 9:22      7358015859\n8/03/2015 4:27      7354221274\n8/03/2015 4:07      7346810719\n7/03/2015 4:25      7339695326\n7/03/2015 4:25      7339698276\n6/03/2015 13:08     7339701226\n6/03/2015 13:04     7317905872\n6/03/2015 12:44     7309771372\n6/03/2015 12:24     7300851599\n6/03/2015 12:04     7292557469\n6/03/2015 11:39     7283439433\n6/03/2015 11:00     7278056128\n6/03/2015 10:41     7267320628\n6/03/2015 10:35     7265158228\n6/03/2015 10:21     7265158228\n6/03/2015 10:01     7255260402\n6/03/2015 7:06      7246047762\n5/03/2015 15:39     7245760885\n5/03/2015 13:41     7170760885\n4/03/2015 21:21     5595760885\n4/03/2015 18:59     5590496476\n4/03/2015 18:59     5585251201"""
    
    # For each line in the dataset, print the row which will be added to the
    # Google Chart.
    for line in DATASET.splitlines():
        date, value = line.split("    ")
    
        # First we parse the date string using strptime(), and write it back out
        # as a Date() constructor to use in JavaScript
        date_obj = datetime.strptime(date, "%d/%m/%Y %H:%M")
        js_date = date_obj.strftime("new Date(%Y, %m, %d, %H, %M, %s, %f)")
    
        js_value = value.strip()
    
        print "[%s, %s]," % (js_date, js_value)
    

Here’s a few rows of the output:

    
    
    [new Date(2015, 03, 10, 10, 55, 1425984900, 000000), 7385498415],
    [new Date(2015, 03, 10, 09, 15, 1425978900, 000000), 7379639094],
    [new Date(2015, 03, 10, 08, 55, 1425977700, 000000), 7376777516],
    [new Date(2015, 03, 10, 08, 35, 1425976500, 000000), 7368304217],
    

Then I took the [scatter plot
example](https://developers.google.com/chart/interactive/docs/gallery/scatterchart)
from Google’s Chart gallery, dropped in the date values I got from the script
above, and changed a few of the options. This is what it looks like:

![enter image description here](http://i.stack.imgur.com/2spPA.png)

and here’s the HTML:

    
    
    <head>
      <!--Load the AJAX API-->
      <script type="text/javascript" src="https://www.google.com/jsapi"></script>
      <script>
        google.load("visualization", "1", {packages:["corechart"]});
        google.setOnLoadCallback(drawChart);
        function drawChart() {
          var data = google.visualization.arrayToDataTable([
            ['Date', 'Value'],
            [new Date(2015, 03, 10, 10, 55), 7385498415],
            [new Date(2015, 03, 10, 09, 15), 7379639094],
            [new Date(2015, 03, 10, 08, 55), 7376777516],
            [new Date(2015, 03, 10, 08, 35), 7368304217],
            [new Date(2015, 03, 10, 08, 12), 7358015859],
            [new Date(2015, 03, 08, 09, 22), 7358015859],
            [new Date(2015, 03, 08, 04, 27), 7354221274],
            [new Date(2015, 03, 08, 04, 07), 7346810719],
            [new Date(2015, 03, 07, 04, 25), 7339695326],
            [new Date(2015, 03, 07, 04, 25), 7339698276],
            [new Date(2015, 03, 06, 13, 08), 7339701226],
            [new Date(2015, 03, 06, 13, 04), 7317905872],
            [new Date(2015, 03, 06, 12, 44), 7309771372],
            [new Date(2015, 03, 06, 12, 24), 7300851599],
            [new Date(2015, 03, 06, 12, 04), 7292557469],
            [new Date(2015, 03, 06, 11, 39), 7283439433],
            [new Date(2015, 03, 06, 11, 00), 7278056128],
            [new Date(2015, 03, 06, 10, 41), 7267320628],
            [new Date(2015, 03, 06, 10, 35), 7265158228],
            [new Date(2015, 03, 06, 10, 21), 7265158228],
            [new Date(2015, 03, 06, 10, 01), 7255260402],
            [new Date(2015, 03, 06, 07, 06), 7246047762],
            [new Date(2015, 03, 05, 15, 39), 7245760885],
            [new Date(2015, 03, 05, 13, 41), 7170760885],
            [new Date(2015, 03, 04, 21, 21), 5595760885],
            [new Date(2015, 03, 04, 18, 59), 5590496476],
            [new Date(2015, 03, 04, 18, 59), 5585251201]
          ]);
    
          var options = {
            hAxis: {title: 'Date'},
            vAxis: {title: 'Variable'},
            legend: 'none',
            height: 500,
            width: 1100
          };
    
          var chart = new google.visualization.ScatterChart(document.getElementById('chart_div'));
    
          chart.draw(data, options);
        }
      </script>
    </head>
    
    <body>
    
    <div id="chart_div"></div>
    
    </body>

