title: Use pandas to group data daily and weekly based on epoch time
body: <p>I have data like,</p>
  
  <pre><code>[
    {
      "arrivalDate": 1493611200000,
      "price": 4588
    },
    {
      "arrivalDate": 1493352000000,
      "price": 4630
    },
    {
      "arrivalDate": 1493179200000,
      "price": 4553
    },
    {
      "arrivalDate": 1493092800000,
      "price": 4530
    },
    {
      "arrivalDate": 1493006400000,
      "price": 4578
    },
    {
      "arrivalDate": 1490572800000,
      "price": 4457
    }
  ]
  </code></pre>
  
  <p>and I want to aggregate and show graphs for daily prices and weekly average prices.</p>
  
  <p>The daily stats is easy but I am not able to figure out the weekly average. Please note that my epoch is in milliseconds.</p>
  
  <p>I have tried,</p>
  
  <pre><code>def get_daily_weekly_and_monthly_aggregates(datewise_prices):
      p_dataframe = pandas.DataFrame.from_records(datewise_prices)
  
      daily = p_dataframe.sort_values(by=['arrivalDate'], ascending=[False])[:6]  # for 7 days
      # TODO - This should be configurable. Not hardcoded.
      daily['label'] = daily['arrivalDate'].map(lambda x: pytz.timezone('Asia/Kolkata').localize(datetime.fromtimestamp(x/1000)).strftime("%d %b"))
      daily_list = daily.to_dict('records')
  
      pandas.to_datetime(p_dataframe['arrivalDate'], unit='ms')
      p_dataframe.set_index(pandas.DatetimeIndex(p_dataframe["arrivalDate"]), inplace=True)
      weekly = p_dataframe.groupby(pandas.TimeGrouper("W")).agg(lambda grp: list(grp))
      weekly.mean()
      weekly_list = weekly.to_dict('records')
  
      return daily_list, weekly_list
  </code></pre>
  
  <p>I get weekly_list as,</p>
  
  <pre><code>[
    {
      "arrivalDate": [
        1490054400000,
        1490572800000,
        1493006400000,
        1493092800000,
        1493179200000,
        1493352000000,
        1493611200000
      ],
      "modalPrice": [
        4357,
        4457,
        4578,
        4530,
        4553,
        4630,
        4588
      ]
    }
  ]
  </code></pre>
  
  <p>which is not desired. </p>
  
  <p>What I actually need is,</p>
  
  <pre><code>[
      {
          "week": "1 May to 7 May",
          "avg": 4588
      },
      {
          "week": "24 Apr to 30 Apr",
          "avg": 4572
      },
      {
          "week": "27 Mar to 02 Apr",
          "avg": 4457
      }
  ] 
  </code></pre>
  
  <p>How do I do this?</p>