title: How to extract key value pairs into dict if other tags are present?
body: <p>I am trying to extract Key/Value{1,2} pairs from a HTML table with python and pull them into a dictionary.</p>
  
  <p>The table elements look not always the same, which is the reason for my question. </p>
  
  <p>A minimal example:</p>
  
  <pre><code>  &lt;div class="grabme"&gt;
    &lt;table&gt;
       &lt;tbody&gt;
  
          &lt;tr&gt;
             &lt;td colspan="2"&gt;
                &lt;p class="1st 2nd 3rd"&gt;
                   Box Headline&lt;/p&gt;
             &lt;/td&gt;
          &lt;/tr&gt;
  
          &lt;tr&gt;
             &lt;td&gt;&lt;strong&gt;First Key&lt;/strong&gt;&lt;/td&gt;
             &lt;td&gt;&lt;span&gt;Value&lt;/span&gt;&lt;/td&gt;
             &lt;script&gt;
             &lt;/script&gt;
          &lt;/tr&gt;
  
          &lt;tr&gt;
                &lt;td&gt;&lt;strong&gt;2. Key&lt;/strong&gt;&lt;/td&gt;
                &lt;td&gt;&lt;a&gt;Value&lt;/a&gt;&lt;br&gt;&lt;/td&gt;
          &lt;/tr&gt;
  
          &lt;tr&gt;
             &lt;td&gt;&lt;strong&gt;3. Key&lt;/strong&gt;&lt;/td&gt;
             &lt;td&gt;Value&lt;/td&gt;
          &lt;/tr&gt;
  
          &lt;tr&gt;
             &lt;td&gt;&lt;strong&gt;4. Key&lt;/strong&gt;&lt;/td&gt;
             &lt;td&gt;
             &lt;a &gt;Val 1&lt;/a&gt;
                Val 2
  
                &lt;script&gt;
                      $(document).ready(function () {
                         $('.class').click(function (e) {
                            e.bla();
                            sel.bla('/bla/bla', {
                                  bla: true
                               }
                            );
                         });
                      });
                   &lt;/script&gt;
                &lt;/td&gt;
          &lt;/tr&gt;
  
          &lt;tr&gt;
                &lt;td&gt;&lt;strong&gt;5. Key&lt;/strong&gt;&lt;/td&gt;
                &lt;td&gt;
                   &lt;i&gt;&lt;/i&gt;
                   Value
                &lt;/td&gt;
          &lt;/tr&gt;
  
       &lt;/tbody&gt;
  
       &lt;tbody&gt;
          &lt;tr&gt;
             &lt;td colspan="2"&gt;
                &lt;p class=""&gt;
                   Heading 2&lt;/p&gt;
             &lt;/td&gt;
          &lt;/tr&gt;
  
          &lt;tr&gt;
             &lt;td&gt;&lt;strong&gt;6. Key&lt;/strong&gt;&lt;/td&gt;
             &lt;td&gt;Value&lt;/td&gt;
          &lt;/tr&gt;
       &lt;/tbody&gt;
    &lt;/table&gt;
  </code></pre>
  
  <p></p>
  
  <p>Getting the key is easy:</p>
  
  <pre><code>keys = response.xpath('//div[@class="grabme"]/table/tbody/tr/td/strong/text()').extract()
  </code></pre>
  
  <p>Unfortunately I can not get all Keys in the example, as Key 6 is within a new tbody. But as a hack, I could get it seperatly and apend to dict later.</p>
  
  <p>Getting the values is a lot harder. My best shot is this:</p>
  
  <pre><code>values = [remove_tags(w).strip() for w in response.xpath('//div[@class="grabme"]/table/tbody/tr/td[1]/text()').extract()]
  </code></pre>
  
  <p>This unfortunatelly does not work, due to the extra html tags.
  If I would be able to grab all values, then I could pull them into a dict:</p>
  
  <pre><code>dict = {first: second for first, second in zip(keys, values)}
  </code></pre>
  
  <p>This part might also be tricky, as the example shows that Key 4 has 2 Values. it would be OK to put them into one value with a seperator so I can later process accordingly.</p>
  
  <p>How do I get the values in my example or even better, is there a smarter way in getting a dictionary of all the desired key, value pairs?</p>
  
  <p>This attempt failed because of the difference in structure:</p>
  
  <pre><code>cells = response.xpath('//div[@class="grabme"]/tbody/tr/td/text()').extract()
  dict = {first: second for first, second in zip(cells[::2], cells[1::2])}
  </code></pre>