title: How can I scrape career path job titles from this javascript page using Python
body: <p>How can I scrape career path job titles from this javascript page using Python?</p>
  
  <p>'<a href="https://www.dice.com/career-paths?title=PHP%2BDeveloper&amp;location=San%2BDiego,%2BCalifornia,%2BUs,%2BCA&amp;experience=0&amp;sortBy=mostProbableTransition" rel="nofollow noreferrer">https://www.dice.com/career-paths?title=PHP%2BDeveloper&amp;location=San%2BDiego,%2BCalifornia,%2BUs,%2BCA&amp;experience=0&amp;sortBy=mostProbableTransition</a>'</p>
  
  <p>This is my code snippet, the returned soup doesn't have any the text data I need!</p>
  
  <pre class="lang-py prettyprint-override"><code>import requests
  from bs4 import BeautifulSoup
  import json
  import re
  from selenium import webdriver
  from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
  from selenium.webdriver.common.keys import Keys
  from selenium.webdriver.common.by import By
  from selenium.webdriver.support.wait import WebDriverWait
  from selenium.webdriver.support import expected_conditions as EC
  
  
  # get BeautifulSoup object
  def get_soup(url):
      """
      This function returns the BeautifulSoup object.
  
      Parameters:
          url: the link to get soup object for
  
      Returns:
          soup: BeautifulSoup object
      """
      req = requests.get(url)
      soup = BeautifulSoup(req.text, 'html.parser')
      return soup
  
  # get selenium driver object
  def get_selenium_driver():
      """
      This function returns the selenium driver object.
  
      Parameters:
          None
  
      Returns:
          driver: selenium driver object
      """
      options = webdriver.FirefoxOptions()
      options.add_argument('-headless')
  
      driver = webdriver.Firefox(executable_path=r"geckodriver", firefox_options = options)
  
      return driver
  
  # get soup obj using selenium
  def get_soup_using_selenium(url):
      """
      Given the url of a page, this function returns the soup object.
  
      Parameters:
          url: the link to get soup object for
  
      Returns:
          soup: soup object
      """
      options = webdriver.FirefoxOptions()
      options.add_argument('-headless')
  
      driver = webdriver.Firefox(executable_path=r"geckodriver", firefox_options = options)
      driver.get(url)
      driver.implicitly_wait(3)
  
      html = driver.page_source
      soup = BeautifulSoup(html, 'html.parser')
  
      driver.close()
  
      return soup
  
  
  
  
  title = "PHP%2BDeveloper"
  location = "San%2BDiego,%2BCalifornia,%2BUs,%2BCA"
  years_of_experirence = "0"
  sort_by_filter = "mostProbableTransition"
  
  url = "https://www.dice.com/career-paths?title={}&amp;location={}&amp;experience={}&amp;sortBy={}".format(title, location, years_of_experirence , sort_by_filter)
  career_paths_page_soup = get_soup(url)
  
  </code></pre>