title: "Python for Data Analysis" tutorial code runs unpredictably - US Baby Names proportions
body: <p>I'm working through Wes McKinney's <em>Python for Data Analysis</em>. While I'm working in Python 3 and the book is written in Python 2, this is generally not an issue, and if anything a good exercise.</p>
  
  <p>However, I've reached an impasse on Chapter 2, example 3: US Baby Names 1880 - 2010 (pg. 34). The purpose of the following code is to insert a column titled 'prop' that contains the fraction of babies given a name for each year and gender into the dataframe:</p>
  
  <pre><code>def add_prop(group):
      births = group.births.astype(float)
      group['prop']=births/births.sum()
      return group
  
  names=names.groupby(['year','sex']).apply(add_prop)
  </code></pre>
  
  <p>'names' is a dataframe with five columns ('name', 'sex', 'births', 'year', and this adds 'prop') and approximately 1.7 million rows. In order to test whether prop was added correctly, you then test when the proportions sum to approximately 1 with <code>np.allclose(names.groupby(['year','sex']).prop.sum(), 1)</code>.</p>
  
  <p>My problem is that the function runs unpredictably. Perhaps once out of every 15 or 20 runs np.allclose will be true, and the function will have been applied to the dataframe correctly. Otherwise np.allclose is false. Additionally, it's wrong in different ways. Later you use this dataframe to graph the proportion of births represented in the top 1000 names by sex, and the shape of that graph changes constantly. <a href="https://i.stack.imgur.com/yR2a2.png" rel="nofollow noreferrer">Some examples of graph change: <img src="https://i.stack.imgur.com/yR2a2.png" alt="Some examples of graph change"></a> I know the problem is in how proportion is being calculated and added because the rest of the dataframe doesn't vary.</p>
  
  <p>What is introducing unpredictability into this example? While I suspect it's the .apply() command, I'm not sure and don't know how to test my hypothesis. It's been suggested to me that part of the code block is deprecated, but Jupyter Notebook doesn't come up with a warning and I haven't been able to find anything online. I've gone over my code twice, and overall it's virtually identical to the book's, and is identical in the case of this block. Thanks in advance.</p>