title: Extracting JSON from HTML using BeautifulSoup python
body: <p>While I was practicing some web-scraping on a webpage (param cookies required), I found myself having problems to scrape out JSON data embedded in the HTML. The following was what I did:</p>
  <pre><code>import requests from bs4
  import BeautifulSoup as soup
  import json
     
  my_url = 'https://www.lazada.sg/catalog/?spm=a2o42.home.search.1.488d46b5mJGzEu&amp;q=switch%20games&amp;_keyori=ss&amp;from=search_history&amp;sugg=switch%20games_0_1'  
  
  cookies = {
      &quot;Hm_lpvt_7cd4710f721b473263eed1f0840391b4&quot;: &quot;1548175412&quot;,
      &quot;Hm_lvt_7cd4710f721b473263eed1f0840391b4&quot;: &quot;1548140525&quot;,
      &quot;x5sec&quot;:&quot;7b22617365727665722d6c617a6164613b32223a223832333339343739626466613939303562613535386138333266383365326132434c4b516e65494645495474764a322b706f6d6f6941453d227d&quot;, }
  
  ret = requests.get(my_url, cookies=cookies)
  print(&quot;New Super Mario Bros&quot; in ret.text) # True
  
  page_soup = soup(ret.text, 'html.parser')
  data = page_soup.findAll('script', {'type':'application/ld+json'})
   
  </code></pre>
  <p>The output is as follows:</p>
  <pre><code>[
    &lt;script type=&quot;application/ld+json&quot;&gt;{
      &quot;@context&quot;: &quot;https://schema.org&quot;,
      &quot;@type&quot;: &quot;BreadcrumbList&quot;,
      &quot;itemListElement&quot;: [
        {
          &quot;item&quot;: {
            &quot;name&quot;: &quot;Home&quot;,
            &quot;@id&quot;: &quot;https://www.lazada.sg/&quot;
          },
          &quot;@type&quot;: &quot;ListItem&quot;,
          &quot;position&quot;: &quot;1&quot;
        }
      ]
    }&lt;/script&gt;,
    &lt;script type=&quot;application/ld+json&quot;&gt;{
      &quot;@context&quot;: &quot;https://schema.org&quot;,
      &quot;@type&quot;: &quot;ItemList&quot;,
      &quot;itemListElement&quot;: [
        {
          &quot;offers&quot;: {
            &quot;priceCurrency&quot;: &quot;SGD&quot;,
            &quot;@type&quot;: &quot;Offer&quot;,
            &quot;price&quot;: &quot;71.00&quot;,
            &quot;availability&quot;: &quot;https://schema.org/InStock&quot;
          },
          &quot;image&quot;: &quot;https://sg-test-11.slatic.net/p/670a73a9613c36b2bb01555ab4092ba2.jpg&quot;,
          &quot;@type&quot;: &quot;Product&quot;,
          &quot;name&quot;: &quot;Switch: Super Mario Party [Available in Stock! Immediate Shipping]&quot;,
          &quot;url&quot;: &quot;https://www.lazada.sg/products/switch-super-mario-party-available-in-stock-immediate-shipping-i278269540-s429667097.html?search=1&quot;
        },
        ...
  </code></pre>
  <p>I tried to follow an existing thread <a href="https://stackoverflow.com/questions/48906246/extract-json-from-html-in-python-beautifulsoup">Extract json from html in python beautifulsoup</a> but found myself stuck, probably due to the different JSON formatting in the HTML soup. The part which I scrape out contains all the different products in that page, is there a way where I further scrape out each product's details (eg. Title, price, rating, etc) and count the number of products present? Thanks!</p>