title: Why does pandas use "NaN" from numpy, instead of its own null value?
body: <p>This is somewhat of a broad topic, but I will try to pare it to some specific questions.</p>
  <p>In starting to answer questions on SO, I have found myself sometimes running into a silly error like this when making toy data:</p>
  <pre><code>In[0]:
  
  import pandas as pd
  
  df = pd.DataFrame({&quot;values&quot;:[1,2,3,4,5,6,7,8,9]})
  df[df &lt; 5] = np.nan
  
  Out[0]:
  NameError: name 'np' is not defined
  </code></pre>
  <p>I'm so used to automatically importing <code>numpy</code> with <code>pandas</code> that this doesn't usually occur in real code.  However, it did make me wonder why <code>pandas</code> doesn't have it's own value/object for representing null values.</p>
  <p>I only recently realized that you could just use the Python <code>None</code> instead for a similar situation:</p>
  <pre><code>import pandas as pd
  
  df = pd.DataFrame({&quot;values&quot;:[1,2,3,4,5,6,7,8,9]})
  df[df &lt; 5] = None
  </code></pre>
  <p>Which works as expected and doesn't produce an error.  But I have felt like the convention on SO that I have seen is to use <code>np.nan</code>, and that people are usually referring to <code>np.nan</code> when discussing null values (this is perhaps why I hadn't realized <code>None</code> can be used, but maybe that was my own idiosyncrasy).</p>
  <p>Briefly looking into this, I have seen now that <code>pandas</code> <strong>does</strong> have a <code>pandas.NA</code> value since 1.0.0, but <em>I have never seen anyone use it in a post</em>:</p>
  <pre><code>In[0]:
  
  import pandas as pd
  import numpy as np
  
  df = pd.DataFrame({'values':np.random.rand(20,)})
  df['above'] = df['values']
  df['below'] = df['values']
  df['above'][df['values']&gt;0.7] = np.nan
  df['below'][df['values']&lt;0.3] = pd.NA
  
  df['names'] = ['a','b','c','a','b','c','a','b','c','a']*2
  df.loc[df['names']=='a','names'] = pd.NA
  df.loc[df['names']=='b','names'] = np.nan
  df.loc[df['names']=='c','names'] = None
  df
  
  Out[0]:
        values     above     below names
  0   0.323531  0.323531  0.323531  &lt;NA&gt;
  1   0.690383  0.690383  0.690383   NaN
  2   0.692371  0.692371  0.692371  None
  3   0.259712  0.259712       NaN  &lt;NA&gt;
  4   0.473505  0.473505  0.473505   NaN
  5   0.907751       NaN  0.907751  None
  6   0.642596  0.642596  0.642596  &lt;NA&gt;
  7   0.229420  0.229420       NaN   NaN
  8   0.576324  0.576324  0.576324  None
  9   0.823715       NaN  0.823715  &lt;NA&gt;
  10  0.210176  0.210176       NaN  &lt;NA&gt;
  11  0.629563  0.629563  0.629563   NaN
  12  0.481969  0.481969  0.481969  None
  13  0.400318  0.400318  0.400318  &lt;NA&gt;
  14  0.582735  0.582735  0.582735   NaN
  15  0.743162       NaN  0.743162  None
  16  0.134903  0.134903       NaN  &lt;NA&gt;
  17  0.386366  0.386366  0.386366   NaN
  18  0.313160  0.313160  0.313160  None
  19  0.695956  0.695956  0.695956  &lt;NA&gt;
  </code></pre>
  <p>So it seems that for numerical values, the distinction between these different null values doesn't matter, but they are represented differently for strings (and perhaps for other data types?).</p>
  <p><strong>My questions based on the above</strong>:</p>
  <ul>
  <li>Is it conventional to use <code>np.nan</code> (rather than <code>None</code>) to represent null values in <code>pandas</code>?</li>
  <li>Why did <code>pandas</code> not have its own null value for most of its lifetime (until last year)?  What was the motivation for adding?</li>
  <li>In cases where you can have multiple types of missing values in one <code>Series</code> or column, is there any difference between them?  Why are they not represented identically (as with numerical data)?</li>
  </ul>
  <p>I fully anticipate that I may have a flawed interpretation of things and the distinction between <code>pandas</code> and <code>numpy</code>, so please correct me.</p>