title: Pandas: Dynamically add a row and columns and input values to it
body: <p>I am working with a large dataset that iteratively fetches n number of child URLs for a particular parent URL. </p>
  
  <p>I initially used excel to record the data (test the working my code actually). But later found out that the idea is not worth it as the output data were huge.</p>
  
  <p>for example:
  i have two set of data:</p>
  
  <pre><code>amazon.com: ['a','b','c','d','e']
  a         : ['k','j','e','f']
  </code></pre>
  
  <ul>
  <li>Here in the first case, <code>amazon.com</code> is the parent URL and the list of values are it's child URLs. </li>
  <li>In the next case <code>a</code> becomes the parent URL and the list of values are it's child URLs.</li>
  </ul>
  
  <p>Now what I actually require is to get a dataframe like:</p>
  
  <pre><code>               a    b    c    d    e    k    j    f
   amazon.com    1    1    1    1    1
       a                             1    1    1    1
  </code></pre>
  
  <p>where 1 can be assumed to be a value to show that say <code>a is the child of amazon.com</code></p>
  
  <p>Now the problem is I won't have the data as shown above. They are obtained dynamically as I crawl through the website.</p>
  
  <p>So the flow would be:</p>
  
  <pre><code>Open a website URL
  records the URL (parent URL - this is where we get the URL)
  records all the URLs present in the page (child URL - this is where we get all the child URLs corresponding to the parent URL and hence can populate our list/dictionary and hence the dataframe)
  </code></pre>
  
  <p>As can be noticed, no duplicates column headers are found.</p>
  
  <p>Can someone help me out on this one?</p>