title: Scientific Computing & Ipython Notebook: How to organize code?
body: <p>I'm using Ipython Notebook to my research. As my file grows bigger, I constantly extract code out, things like plot method, fitting method etc. </p>
  
  <p>I think I need a way to organize this. Is there any good way to do it??</p>
  
  <hr>
  
  <p>Currently, I do this by:</p>
  
  <pre><code>data/
  helpers/
  my_notebook.ipynb
  import_file.py
  </code></pre>
  
  <p>I store data at <code>data/</code>, and extract <code>helper method</code> into <code>helpers/</code>, and divide them into files like <code>plot_helper.py</code>, <code>app_helper.py</code>, etc.</p>
  
  <p>I summarize the imports in <code>import_file.py</code>, </p>
  
  <pre><code>from IPython.display import display
  
  import numpy as np
  import scipy as sp
  import pandas as pd
  import matplotlib as mpl
  from matplotlib import pyplot as plt
  import sklearn
  import re
  </code></pre>
  
  <p>And then I can import everything I need in <code>.ipynb</code> at top cell as </p>
  
  <p><a href="https://i.stack.imgur.com/XHbi3.png" rel="noreferrer"><img src="https://i.stack.imgur.com/XHbi3.png" alt="enter image description here"></a></p>
  
  <p>The structure can be seen at <a href="https://github.com/cqcn1991/Wind-Speed-Analysis" rel="noreferrer">https://github.com/cqcn1991/Wind-Speed-Analysis</a></p>
  
  <p>One problem I have right now is that I have too many submodule at <code>helpers/</code>, and it's hard to think which method should be put into which file. </p>
  
  <p>I think a possible way is to organize in <code>pre-processing</code>, <code>processing</code>, <code>post-processing</code>.</p>
  
  <p>UPDATE: </p>
  
  <p>My big jupyter research notebook:
  <a href="https://cdn.rawgit.com/cqcn1991/Wind-Speed-Analysis/master/output_HTML/marham.html" rel="noreferrer">https://cdn.rawgit.com/cqcn1991/Wind-Speed-Analysis/master/output_HTML/marham.html</a></p>
  
  <p>The top cell is <code>standard import</code> + <code>magic</code> + <code>extentions</code></p>
  
  <pre><code>%matplotlib inline
  %load_ext autoreload
  %autoreload 2
  
  from __future__ import division
  from import_file import *
  load_libs()
  </code></pre>