Extracting data with Headless Selenium browser with Python
Question: I'm trying to find a way to extract the GA datalayer object from a website
with a headless browser via Python so therefore; I've follow the instructions
from [here](https://realpython.com/blog/python/headless-selenium-testing-with-
python-and-phantomjs/) and...

    
    
    from selenium import webdriver
    
    driver = webdriver.PhantomJS()
    driver.get("http://www.example.com/")
    # DL = currentURL.dataLayer 
    # Do something with DL
    

What is the keyword I'm looking for here? `driver.something` should give me
the dataLayer object. As a reminder: dataLayer object is a javascript object.


Answer: Since, from what I understand, `dataLayer` is a global variable, use
[`execute_script()`](https://selenium-
python.readthedocs.org/api.html#selenium.webdriver.remote.webdriver.WebDriver.execute_script):

    
    
    driver.execute_script("return dataLayer;")
    

Note, that you may need to [explicitly wait](https://selenium-
python.readthedocs.org/waits.html#explicit-waits) for the page to load before
executing the script.

Or, at least, increase the [page load timeout](https://selenium-
python.readthedocs.org/api.html#selenium.webdriver.remote.webdriver.WebDriver.set_page_load_timeout):

    
    
    driver.set_page_load_timeout(10)
    

