title: How to scrape the response of the files that appear in the Network tab in the Inspect element of a website?
body: <p><a href="https://i.stack.imgur.com/mvRle.png" rel="nofollow noreferrer">Here</a> is a screenshot of the response I am trying to scrape. I am trying to scrape the response that is received on <strong>restaurant</strong> file (document type). I am trying to scrape using <code>selenium</code> and Python.</p>
  <p>I tried different ways to scrape it but couldn't find a particular way.
  I tried using <code>desired_capabilities</code> to get the logs:</p>
  <pre><code>desired_capabilities[&quot;goog:loggingPrefs&quot;] = {&quot;performance&quot;: &quot;ALL&quot;}
  </code></pre>
  <p>The closest I can get to extracting the info about the restaurant response is in the code below, which prints out the info about the restaurant file. However, still I am unable to get the response of the document.</p>
  <p>I am trying to scrape <a href="https://food.grab.com/ph/en/restaurants" rel="nofollow noreferrer">this website</a>.</p>
  <pre><code>import time
  import json
  
  from selenium import webdriver
  from selenium.webdriver.common.action_chains import ActionChains
  from selenium.webdriver.common.keys import Keys
  from selenium.webdriver import DesiredCapabilities
  
  # make chrome log requests
  
  def get_actions(driver):
      actions = ActionChains(driver)
      return actions
  
  def setup():
      desired_capabilities = DesiredCapabilities.CHROME
      desired_capabilities[&quot;goog:loggingPrefs&quot;] = {&quot;performance&quot;: &quot;ALL&quot;}
      driver = webdriver.Chrome(
          desired_capabilities=desired_capabilities, executable_path=r&quot;C:\Users\visha\PycharmProjects\grabScrapper\chromedriver.exe&quot;
      )
  
      address = 'Manila '
      driver.get('https://food.grab.com/ph/en/restaurants')
      time.sleep(2)  # Let the user actually see something!
      driver.find_element_by_class_name(&quot;textPlaceholder___1yEAK&quot;).click()
  
      actions = get_actions(driver)
      actions.send_keys(address)
      actions.perform()
      time.sleep(1)
  
      actions = get_actions(driver)
      actions.send_keys(Keys.ARROW_DOWN)
      actions.perform()
      time.sleep(1)
  
      actions = get_actions(driver)
      actions.send_keys(Keys.ENTER)
      actions.perform()
  
      time.sleep(5)
      driver.refresh()
      time.sleep(5)
  
      logs = driver.get_log(&quot;performance&quot;)
  
      print(json.loads(logs[0][&quot;message&quot;])[&quot;message&quot;])
  
  
  
  if __name__ == '__main__':
      driver = setup()
  </code></pre>
  <p>Here is the output:</p>
  <pre><code>{'method': 'Network.requestWillBeSent', 'params': {'documentURL': 
  'https://food.grab.com/ph/en/restaurants', 'frameId': '511A291FBAE6A23A8A6CC0151F478245',         
  'hasUserGesture': False, 'initiator': {'type': 'other'}, 'loaderId': 
  '398F451EBDC9243DC9A05E18809AD020', 'request': {'headers': {'Upgrade-Insecure-Requests': '1', 
  'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like 
  Gecko) Chrome/93.0.4577.82 Safari/537.36', 'sec-ch-ua': '&quot;Google Chrome&quot;;v=&quot;93&quot;, &quot; Not;A 
  Brand&quot;;v=&quot;99&quot;, &quot;Chromium&quot;;v=&quot;93&quot;', 'sec-ch-ua-mobile': '?0', 'sec-ch-ua-platform': 
  '&quot;Windows&quot;'}, 'initialPriority': 'VeryHigh', 'isSameSite': True, 'method': 'GET', 
  'mixedContentType': 'none', 'referrerPolicy': 'strict-origin-when-cross-origin', 'url': 
  'https://food.grab.com/ph/en/restaurants'}, 'requestId': '398F451EBDC9243DC9A05E18809AD020', 
  'timestamp': 75113.949225, 'type': 'Document', 'wallTime': 1632732378.366389}}
  </code></pre>
  <p>Please help me to scrape the response data of the restaurant file using Python so that I can do some manipulation on my own.</p>