title: Scaling and fitting to a log-normal distribution using a logarithmic axis in python
body: <p>I have a log-normal distributed set of samples. I can visualize the samples using a histrogram with either linear or logarithmic x-axis. I can perform a fit to the histogram to get the PDF and then scale it to the histrogram in the plot with the linear x-axis, see also <a href="https://stackoverflow.com/questions/34893615/scaling-the-fitted-pdf-of-a-log-normal-distribution-to-the-histrogram-in-python/">this previously posted question</a>.</p>
  
  <p>I am, however, not able to properly plot the PDF into the plot with the logarithmic x-axis. </p>
  
  <p>Unfortunately, it is not only a problem with the scaling of the area of the PDF to the histogram but the PDF is also shifted to the left, as you can see from the following plot.</p>
  
  <p><a href="https://i.stack.imgur.com/gizkR.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/gizkR.png" alt="Histogram and fitted and scaled PDF with a linear x-axis (left) and a logarithmic x-axis (right)"></a></p>
  
  <p>My question now is, what am I doing wrong here? Using the CDF to plot the expected histogram, <a href="https://stackoverflow.com/a/34896229/838992">as suggested in this answer</a>, works. I would just like to know what I am doing wrong in this code as in my understanding it should work too.</p>
  
  <p>This is the python code (I am sorry that it is rather long but I wanted to post a "full stand-alone version"):</p>
  
  <pre><code>import numpy as np
  import matplotlib.pyplot as plt
  import scipy.stats
  
  # generate log-normal distributed set of samples
  np.random.seed(42)
  samples   = np.random.lognormal( mean=1, sigma=.4, size=10000 )
  
  # make a fit to the samples
  shape, loc, scale = scipy.stats.lognorm.fit( samples, floc=0 )
  x_fit       = np.linspace( samples.min(), samples.max(), 100 )
  samples_fit = scipy.stats.lognorm.pdf( x_fit, shape, loc=loc, scale=scale )
  
  # plot a histrogram with linear x-axis
  plt.subplot( 1, 2, 1 )
  N_bins = 50
  counts, bin_edges, ignored = plt.hist( samples, N_bins, histtype='stepfilled', label='histogram' )
  # calculate area of histogram (area under PDF should be 1)
  area_hist = .0
  for ii in range( counts.size):
      area_hist += (bin_edges[ii+1]-bin_edges[ii]) * counts[ii]
  # oplot fit into histogram
  plt.plot( x_fit, samples_fit*area_hist, label='fitted and area-scaled PDF', linewidth=2)
  plt.legend()
  
  # make a histrogram with a log10 x-axis
  plt.subplot( 1, 2, 2 )
  # equally sized bins (in log10-scale)
  bins_log10 = np.logspace( np.log10( samples.min()  ), np.log10( samples.max() ), N_bins )
  counts, bin_edges, ignored = plt.hist( samples, bins_log10, histtype='stepfilled', label='histogram' )
  # calculate area of histogram
  area_hist_log = .0
  for ii in range( counts.size):
      area_hist_log += (bin_edges[ii+1]-bin_edges[ii]) * counts[ii]
  # get pdf-values for log10 - spaced intervals
  x_fit_log       = np.logspace( np.log10( samples.min()), np.log10( samples.max()), 100 )
  samples_fit_log = scipy.stats.lognorm.pdf( x_fit_log, shape, loc=loc, scale=scale )
  # oplot fit into histogram
  plt.plot( x_fit_log, samples_fit_log*area_hist_log, label='fitted and area-scaled PDF', linewidth=2 )
  
  plt.xscale( 'log' )
  plt.xlim( bin_edges.min(), bin_edges.max() )
  plt.legend()
  plt.show()
  </code></pre>
  
  <p><strong>Update 1</strong>: </p>
  
  <p>I forgot to mention the versions I am using:</p>
  
  <pre><code>python      2.7.6
  numpy       1.8.2
  matplotlib  1.3.1
  scipy       0.13.3
  </code></pre>
  
  <p><strong>Update 2</strong>:</p>
  
  <p>As pointed out by @Christoph and @zaxliu (thanks to both), the problem lies in the scaling of the PDF. It works when I am using the same bins as for the histogram, as in @zaxliu's solution, but I still have some problems when using a higher resolution for the PDF (as in my example above). This is shown in the following figure:</p>
  
  <p><a href="https://i.stack.imgur.com/aD2gV.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/aD2gV.png" alt="Histogram and fitted and scaled PDF with a linear x-axis (left) and a logarithmic x-axis (right)"></a></p>
  
  <p>The code for the figure on the right hand side is (I left out the import and data-sample generation stuff, which you can find both in the above example):</p>
  
  <pre><code># equally sized bins in log10-scale
  bins_log10 = np.logspace( np.log10( samples.min()  ), np.log10( samples.max() ), N_bins )
  counts, bin_edges, ignored = plt.hist( samples, bins_log10, histtype='stepfilled', label='histogram' )
  
  # calculate length of each bin (required for scaling PDF to histogram)
  bins_log_len = np.zeros( bins_log10.size )
  for ii in range( counts.size):
      bins_log_len[ii] = bin_edges[ii+1]-bin_edges[ii]
  
  # get pdf-values for same intervals as histogram
  samples_fit_log = scipy.stats.lognorm.pdf( bins_log10, shape, loc=loc, scale=scale )
  
  # oplot fitted and scaled PDF into histogram
  plt.plot( bins_log10, np.multiply(samples_fit_log,bins_log_len)*sum(counts), label='PDF using histogram bins', linewidth=2 )
  
  # make another pdf with a finer resolution
  x_fit_log       = np.logspace( np.log10( samples.min()), np.log10( samples.max()), 100 )
  samples_fit_log = scipy.stats.lognorm.pdf( x_fit_log, shape, loc=loc, scale=scale )
  # calculate length of each bin (required for scaling PDF to histogram)
  # in addition, estimate middle point for more accuracy (should in principle also be done for the other PDF)
  bins_log_len       = np.diff( x_fit_log )
  samples_log_center = np.zeros( x_fit_log.size-1 )
  for ii in range( x_fit_log.size-1 ):
      samples_log_center[ii] = .5*(samples_fit_log[ii] + samples_fit_log[ii+1] )
  
  # scale PDF to histogram
  # NOTE: THIS IS NOT WORKING PROPERLY (SEE FIGURE)
  pdf_scaled2hist = np.multiply(samples_log_center,bins_log_len)*sum(counts)
  
  # oplot fit into histogram
  plt.plot( .5*(x_fit_log[:-1]+x_fit_log[1:]), pdf_scaled2hist, label='PDF using own bins', linewidth=2 )
  
  plt.xscale( 'log' )
  plt.xlim( bin_edges.min(), bin_edges.max() )
  plt.legend(loc=3)
  </code></pre>