title: Scraping a website that uses AJAX Post to populate the data
body: <p>I was wondering if it was at all possible using <code>beautifulsoup</code> to scrape a website that loads a table based on an ajax call.</p>
  
  <p>Below is the python code I am using to access the div that contains the table</p>
  
  <p><code>table = bs.find(lambda tag: tag.name=='div' and tag.has_key('id') and tag['id']=="id+name")</code></p>
  
  <p>When that is executed I get an empty div <code>&lt;div id="id+name"&gt;&lt;/div&gt;</code></p>
  
  <p>The java script/ajax function looks like this</p>
  
  <pre><code>function getTable(){
      $.ajax({
          type: "POST",
          url: "&lt;some processing file .asmx&gt;",
          contentType: "application/json; charset=utf-8",
          dataType:"json",
          success: function(msg){
              $('#table+id').html(msg.d);
          }
      });
  </code></pre>
  
  <p>I think I'm getting blank because it is trying to scrape the table before the page is processed. Is this something that can be handled in beautiful soup?</p>