title: Extracting text from span tag with no classes in BeautifulSoup
body: <p>I'm trying to extract data from a website for the purposes of finishing a small data analysis project. Here is the the HTML source that I'm dealing with (All the divs that I want to extract data from have the same structure exactly).</p>
  <pre><code>url = &quot;https://www.rystadenergy.com/newsevents/news/press-releases/&quot;
  results = requests.get(url)
  soup = BeautifulSoup(results.text, &quot;html.parser&quot;)
  
  
     &lt;div class=&quot;col-12 col-md-6 col-lg-4 mt-3 news-events-list__item&quot; data-category=&quot;Oil Markets&quot; data-month=&quot;11&quot; data-year=&quot;2020&quot;&gt;
       &lt;a class=&quot;d-block bg-light p-3 text-body text-decoration-none h-100&quot; href=&quot;/newsevents/news/press-releases/prices-at-stake-if-opec-increases-output-in-january-a-200-million-barrel-glut-will-build-through-may/&quot;&gt;
        &lt;small class=&quot;mb-3 d-flex flex-wrap justify-content-between&quot;&gt;
         &lt;time datetime=&quot;2020-11-30&quot;&gt;
          November 30, 2020
         &lt;/time&gt;
         &lt;span&gt;
          Oil Markets
         &lt;/span&gt;
        &lt;/small&gt;
        &lt;h5 class=&quot;mb-0&quot;&gt;
         Prices at stake: If OPEC+ increases output in January, a 200 million-barrel glut will build through May
        &lt;/h5&gt;
       &lt;/a&gt;
      &lt;/div&gt;
  </code></pre>
  <p>Fortunately, I succeeded in extracting the titles of the articles and their publishing dates. I've first created <code>bs4.element.ResultSet</code> and then wrote a loop in order to iterate through each date as follow and it worked properly (same happened for the title of the article).</p>
  <pre><code>divs = soup.find_all('div', class_='col-12 col-md-6 col-lg-4 mt-3 news-events-list__item')
  
  dates = []
  for container in divs:
      date = container.find('time')
      dates.append(date['datetime'])
  </code></pre>
  <p>However, when I tried to extract the category of each article, which lives between <code>&lt;span&gt;&lt;/span&gt;</code> (Oil Markets in my case), I've got an <code>error that 'NoneType' object has no attribute 'text</code>. The code I used to do so was:</p>
  <pre><code>for container in divs:
      topic = container.find('span').text
      topics.append(topic)  
  </code></pre>
  <p>The weird thing here is that when I <code>print(topics)</code>, I've got a list contains more elements than the actual ones (almost 800 element and sometimes even more) and the elements were mixed and include strings and bs4 element tags at the same time. Here is a snapshot of the list I've got:</p>
  <pre><code>&lt;/span&gt;, &lt;span&gt; E&amp;amp;P, Oil Markets, Supply Chain &lt;/span&gt;, &lt;span&gt; Oil Markets, Gas Markets &lt;/span&gt;, &lt;span&gt; Supply Chain &lt;/span&gt;, &lt;span&gt; Gas Markets &lt;/span&gt;, &lt;span&gt; E&amp;amp;P &lt;/span&gt;, &lt;span&gt; Shale &lt;/span&gt;, &lt;span&gt; Corporate &lt;/span&gt;, &lt;span&gt; E&amp;amp;P &lt;/span&gt;, &lt;span&gt; Oil Markets &lt;/span&gt;, &lt;span&gt; Supply Chain, Other, Renewables &lt;/span&gt;, &lt;span&gt; Gas Markets &lt;/span&gt;, &lt;span&gt; Oil Markets &lt;/span&gt;, &lt;span&gt; Gas Markets &lt;/span&gt;, &lt;span&gt; Gas Markets &lt;/span&gt;, &lt;span&gt; E&amp;amp;P &lt;/span&gt;, &lt;span&gt; Gas Markets &lt;/span&gt;, &lt;span&gt; E&amp;amp;P &lt;/span&gt;, &lt;span&gt; Supply Chain &lt;/span&gt;, &lt;span&gt; Shale &lt;/span&gt;, None, &lt;span&gt; Corporate &lt;/span&gt;, &lt;span&gt; Shale &lt;/span&gt;, None, &lt;span&gt; Renewables &lt;/span&gt;, &lt;span&gt; Renewables &lt;/span&gt;, &lt;span&gt; Renewables &lt;/span&gt;, &lt;span&gt; E&amp;amp;P &lt;/span&gt;, &lt;span&gt; E&amp;amp;P &lt;/span&gt;, &lt;span&gt; E&amp;amp;P &lt;/span&gt;, &lt;span&gt; E&amp;amp;P &lt;/span&gt;, &lt;span&gt; Oil Markets &lt;/span&gt;, &lt;span&gt; E&amp;amp;P &lt;/span&gt;, &lt;span&gt; Supply Chain &lt;/span&gt;, ' Oil Markets ', ' Oil Markets ', ' Supply Chain, Renewables ', ' Oil Markets ', ' Renewables ', ' E&amp;P ', ' Renewables ', ' Supply Chain ', ' Shale ', ' E&amp;P ', ' Shale ', ' Gas Markets ', ' Gas Markets ', ' Supply Chain ', ' Oil Markets ', ' Shale ', ' Oil Markets ', ' Corporate, Oil Markets, Other ', ' Shale ', ' Renewables ', ' Shale ', ' Supply Chain ',
  </code></pre>
  <p>My aim is to extract the categories as a list of strings (they should be 207 categories combined) in order to populate them later in a data frame along with dates and title.</p>
  <p>I've tried the solutions <a href="https://stackoverflow.com/questions/52783383/beautiful-soup-extract-text-between-span-tags">here</a> and <a href="https://stackoverflow.com/questions/52656771/how-to-extract-text-from-span-surrounded-by-div-using-beautifulsoup">here</a> and <a href="https://stackoverflow.com/questions/62428630/python-beautifulsoup-unable-to-extract-span-value">here</a> but with no success. I was wondering if someone can help me to fix this problem.</p>