title: Pandas datetime week not as expected
body: <p>When working with Pandas datetimes, I'm trying to group data by the week and year. However, I have noticed some years where the last day of the year ends up grouped with the first week of the same year. </p>
  
  <pre class="lang-py prettyprint-override"><code>import pandas as pd
  day_df = pd.DataFrame(index=pd.date_range('2016-01-01', '2020-12-31'))
  
  for (week, year), subset in day_df.groupby([day_df.index.week, day_df.index.year]):
       if week == 1:
           print('Week:', subset.index.min(), subset.index.max())
  
  Week: 1 2016-01-04 00:00:00 2016-01-10 00:00:00
  Week: 1 2017-01-02 00:00:00 2017-01-08 00:00:00
  Week: 1 2018-01-01 00:00:00 2018-12-31 00:00:00
  Week: 1 2019-01-01 00:00:00 2019-12-31 00:00:00
  Week: 1 2020-01-01 00:00:00 2020-01-05 00:00:00
  </code></pre>
  
  <p>For 2018 and 2019, the first day of the year ends up grouped with the final day of the year! Is this behavior expected? Why would the final day of the year be week 1? </p>
  
  <p>I've gotten the result I want with a basic <code>if</code> statement, but this <code>week</code> behavior seems like it could lead to issues because it's unexpected.</p>
  
  <p>This does what I intended with the grouping:</p>
  
  <pre class="lang-py prettyprint-override"><code>for (week, year), subset in day_df.groupby([day_df.index.week, day_df.index.year]):
      # Prevent first week of year from including final days of same year
      if set(subset.index.month.unique()) == set([1, 12]):
          subset = subset.loc[subset.index.month == 1]
      if week == 1:
          print('Week:', week, subset.index.min(), subset.index.max())
  
  Week: 1 2016-01-04 00:00:00 2016-01-10 00:00:00
  Week: 1 2017-01-02 00:00:00 2017-01-08 00:00:00
  Week: 1 2018-01-01 00:00:00 2018-01-07 00:00:00
  Week: 1 2019-01-01 00:00:00 2019-01-06 00:00:00
  Week: 1 2020-01-01 00:00:00 2020-01-05 00:00:00
  </code></pre>