title: Requests Module not fetching full website in Python
body: <p>Sorry for a Noob question.... I have written a code which searches google for an image stored locally on my computer. I accomplished this using the requests module. I want to scrape the result page for information about the image but request module never fetches the entire page. It only fetches a part of it and thus I am not able to scrape the website for results</p>
  <pre><code>import requests
  import webbrowser
  from bs4 import BeautifulSoup
  
  filePath = &quot;C:\\Users\\mjjha\\Documents\\Checkrow\\monaLisa.jpg&quot;
  searchUrl = 'http://www.google.com/searchbyimage/upload'
  multipart = {'encoded_image': (filePath, open(filePath, 'rb')), 'image_content': ''}
  response = requests.post(searchUrl, files=multipart, allow_redirects=False)
  fetchUrl = response.headers['Location']
  
  r=requests.get(fetchUrl)
  webbrowser.open(fetchUrl)
  soup=BeautifulSoup(r.content,'html.parser')
  head=soup.find_all('a')
  for i in head:
      print(i['href'])
  </code></pre>
  <p>The web page looks like this:
  <a href="https://i.stack.imgur.com/N9wcg.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/N9wcg.png" alt="enter image description here" /></a></p>
  <p>but when I scrape it for anchor tag links using beautiful soup I get the following result:</p>
  <pre><code>http://www.google.co.in/imghp?hl=en&amp;tab=wi
  http://maps.google.co.in/maps?hl=en&amp;tab=wl
  https://play.google.com/?hl=en&amp;tab=w8
  http://www.youtube.com/?gl=IN&amp;tab=w1
  https://news.google.com/?tab=wn
  https://mail.google.com/mail/?tab=wm
  https://drive.google.com/?tab=wo
  https://www.google.co.in/intl/en/about/products?tab=wh
  http://www.google.co.in/history/optout?hl=en
  /preferences?hl=en
  https://accounts.google.com/ServiceLogin?hl=en&amp;passive=true&amp;continue=http://www.google.hr/webhp%3Ftbs%3Dsbi:AMhZZisFTqPOZmEYpGB89rRLg4R4TfmF3WVQ_1gHEFiENQ8wbYqQq7-KsJUE5KuuxvINd0hFo10EMmP4RzWvOBvRxmsHZ7vm6etW-I36-QfCwmwir1NawORzsWZJffCnSwTpdts39mmQ1EfkcH0R8eGsiJ4_1Xw9DA_1C9mqLpChwRYdgOT-oFNcpt2O25Zhmo6ouG2XA5ZelCbKAChT4DJfGz0TXphXB_1dGEluDV6R_15n42URKCX5Q1zIqR6_16CR0rgXBphz95FMrETLqtPURRbAaWzauYisnSk6jF_1T5GbuoJKHtqThXevhogUSW9ERfZr5vbbWI6DA9c&amp;ec=GAZAAQ
  /search?ie=UTF-8&amp;q=Anne+Frank&amp;oi=ddle&amp;ct=236393864&amp;hl=en-GB&amp;si=AC1wQDDagiMg03ncxeOQZbwVe-CJxRCchC-jr2hCPTxjc9wbgOxFdg4PkIAWeA8WhyCLGGzkibRoi5B84SONt2NaUNMtZff0HVDXAtNUKeMfxbgImvSIzyY%3D&amp;sa=X&amp;ved=0ahUKEwjT1qipk8f4AhUCTmwGHZy2BLcQPQgD
  /advanced_search?hl=en-IN&amp;authuser=0
  http://www.google.hr/setprefs?sig=0_jZL2NlEWh9JZhydIGUbq3LjMUs0%3D&amp;hl=hr&amp;source=homepage&amp;sa=X&amp;ved=0ahUKEwjT1qipk8f4AhUCTmwGHZy2BLcQ2ZgBCAU    
  http://www.google.hr/setprefs?sig=0_jZL2NlEWh9JZhydIGUbq3LjMUs0%3D&amp;hl=hi&amp;source=homepage&amp;sa=X&amp;ved=0ahUKEwjT1qipk8f4AhUCTmwGHZy2BLcQ2ZgBCAY    
  http://www.google.hr/setprefs?sig=0_jZL2NlEWh9JZhydIGUbq3LjMUs0%3D&amp;hl=bn&amp;source=homepage&amp;sa=X&amp;ved=0ahUKEwjT1qipk8f4AhUCTmwGHZy2BLcQ2ZgBCAc    
  http://www.google.hr/setprefs?sig=0_jZL2NlEWh9JZhydIGUbq3LjMUs0%3D&amp;hl=te&amp;source=homepage&amp;sa=X&amp;ved=0ahUKEwjT1qipk8f4AhUCTmwGHZy2BLcQ2ZgBCAg    
  http://www.google.hr/setprefs?sig=0_jZL2NlEWh9JZhydIGUbq3LjMUs0%3D&amp;hl=mr&amp;source=homepage&amp;sa=X&amp;ved=0ahUKEwjT1qipk8f4AhUCTmwGHZy2BLcQ2ZgBCAk    
  http://www.google.hr/setprefs?sig=0_jZL2NlEWh9JZhydIGUbq3LjMUs0%3D&amp;hl=ta&amp;source=homepage&amp;sa=X&amp;ved=0ahUKEwjT1qipk8f4AhUCTmwGHZy2BLcQ2ZgBCAo
  http://www.google.hr/setprefs?sig=0_jZL2NlEWh9JZhydIGUbq3LjMUs0%3D&amp;hl=gu&amp;source=homepage&amp;sa=X&amp;ved=0ahUKEwjT1qipk8f4AhUCTmwGHZy2BLcQ2ZgBCAs    
  http://www.google.hr/setprefs?sig=0_jZL2NlEWh9JZhydIGUbq3LjMUs0%3D&amp;hl=kn&amp;source=homepage&amp;sa=X&amp;ved=0ahUKEwjT1qipk8f4AhUCTmwGHZy2BLcQ2ZgBCAw    
  http://www.google.hr/setprefs?sig=0_jZL2NlEWh9JZhydIGUbq3LjMUs0%3D&amp;hl=ml&amp;source=homepage&amp;sa=X&amp;ved=0ahUKEwjT1qipk8f4AhUCTmwGHZy2BLcQ2ZgBCA0
  http://www.google.hr/setprefs?sig=0_jZL2NlEWh9JZhydIGUbq3LjMUs0%3D&amp;hl=pa&amp;source=homepage&amp;sa=X&amp;ved=0ahUKEwjT1qipk8f4AhUCTmwGHZy2BLcQ2ZgBCA4    
  /intl/en/ads/
  http://www.google.co.in/services/
  /intl/en/about.html
  http://www.google.hr/setprefdomain?prefdom=US&amp;sig=K_fWA_BOpuaXy87gYOc9cg4jE6KwU%3D
  /intl/en/policies/privacy/
  /intl/en/policies/terms/
  PS C:\Users\mjjha\Documents\Checkrow&gt; python -u &quot;c:\Users\mjjha\Documents\Checkrow\1.py&quot;
  http://www.google.co.in/imghp?hl=en&amp;tab=wi
  http://maps.google.co.in/maps?hl=en&amp;tab=wl
  https://play.google.com/?hl=en&amp;tab=w8
  http://www.youtube.com/?gl=IN&amp;tab=w1
  https://news.google.com/?tab=wn
  https://mail.google.com/mail/?tab=wm
  https://drive.google.com/?tab=wo
  https://www.google.co.in/intl/en/about/products?tab=wh
  http://www.google.co.in/history/optout?hl=en
  /preferences?hl=en
  https://accounts.google.com/ServiceLogin?hl=en&amp;passive=true&amp;continue=http://www.google.hr/webhp%3Ftbs%3Dsbi:AMhZZiuVpzNQ_10ZbUunarcxtcBADLP2GPHTlIiPAtNpsOezRg48a1S4ofT8Df9C4NRT_1PzMk4baDStlOQGRp2okPHCALA5TvLodRqGj_1Q9_19KWykusySDjQNkbi67Ob6Kx7LZ0ybQ59c9mvyda27CBq8_19XutgXXgl4hGCLdXX9M3Od0WI9BckSHxv_1zajCMhKj1XaLKl9p7T9S0hfQbyZs4zQNcudXEk_1y3Zle6anU1rmSpEdpeCXC6r_1HnTnTLAtYWQlVFVF6QuCT8W5djGXGXwTjQH7NgkXnOi6q7v4F_1DqTVytnSAcBX6rc1eJFlXwIR2dzR73cs983mzb686VgOqUUNS1IG8w&amp;ec=GAZAAQ
  /search?ie=UTF-8&amp;q=Anne+Frank&amp;oi=ddle&amp;ct=236393864&amp;hl=en-GB&amp;si=AC1wQDDagiMg03ncxeOQZbwVe-CJxRCchC-jr2hCPTxjc9wbgOxFdg4PkIAWeA8WhyCLGGzkibRoi5B84SONt2NaUNMtZff0HVDXAtNUKeMfxbgImvSIzyY%3D&amp;sa=X&amp;ved=0ahUKEwibiJivk8f4AhVaTmwGHenOD1wQPQgD
  /advanced_search?hl=en-IN&amp;authuser=0
  http://www.google.hr/setprefs?sig=0_cvjNCWi4vNXxh56c1c4ZqxAqvrQ%3D&amp;hl=hr&amp;source=homepage&amp;sa=X&amp;ved=0ahUKEwibiJivk8f4AhVaTmwGHenOD1wQ2ZgBCAU    
  http://www.google.hr/setprefs?sig=0_cvjNCWi4vNXxh56c1c4ZqxAqvrQ%3D&amp;hl=hi&amp;source=homepage&amp;sa=X&amp;ved=0ahUKEwibiJivk8f4AhVaTmwGHenOD1wQ2ZgBCAY    
  http://www.google.hr/setprefs?sig=0_cvjNCWi4vNXxh56c1c4ZqxAqvrQ%3D&amp;hl=bn&amp;source=homepage&amp;sa=X&amp;ved=0ahUKEwibiJivk8f4AhVaTmwGHenOD1wQ2ZgBCAc    
  http://www.google.hr/setprefs?sig=0_cvjNCWi4vNXxh56c1c4ZqxAqvrQ%3D&amp;hl=te&amp;source=homepage&amp;sa=X&amp;ved=0ahUKEwibiJivk8f4AhVaTmwGHenOD1wQ2ZgBCAg    
  http://www.google.hr/setprefs?sig=0_cvjNCWi4vNXxh56c1c4ZqxAqvrQ%3D&amp;hl=mr&amp;source=homepage&amp;sa=X&amp;ved=0ahUKEwibiJivk8f4AhVaTmwGHenOD1wQ2ZgBCAk    
  http://www.google.hr/setprefs?sig=0_cvjNCWi4vNXxh56c1c4ZqxAqvrQ%3D&amp;hl=ta&amp;source=homepage&amp;sa=X&amp;ved=0ahUKEwibiJivk8f4AhVaTmwGHenOD1wQ2ZgBCAo    
  http://www.google.hr/setprefs?sig=0_cvjNCWi4vNXxh56c1c4ZqxAqvrQ%3D&amp;hl=gu&amp;source=homepage&amp;sa=X&amp;ved=0ahUKEwibiJivk8f4AhVaTmwGHenOD1wQ2ZgBCAs    
  http://www.google.hr/setprefs?sig=0_cvjNCWi4vNXxh56c1c4ZqxAqvrQ%3D&amp;hl=kn&amp;source=homepage&amp;sa=X&amp;ved=0ahUKEwibiJivk8f4AhVaTmwGHenOD1wQ2ZgBCAw    
  http://www.google.hr/setprefs?sig=0_cvjNCWi4vNXxh56c1c4ZqxAqvrQ%3D&amp;hl=ml&amp;source=homepage&amp;sa=X&amp;ved=0ahUKEwibiJivk8f4AhVaTmwGHenOD1wQ2ZgBCA0    
  http://www.google.hr/setprefs?sig=0_cvjNCWi4vNXxh56c1c4ZqxAqvrQ%3D&amp;hl=pa&amp;source=homepage&amp;sa=X&amp;ved=0ahUKEwibiJivk8f4AhVaTmwGHenOD1wQ2ZgBCA4    
  /intl/en/ads/
  http://www.google.co.in/services/
  /intl/en/about.html
  http://www.google.hr/setprefdomain?prefdom=US&amp;sig=K_d19wKMnK5qQH_fmlL2YBuhhR_BE%3D
  /intl/en/policies/privacy/
  /intl/en/policies/terms/
  PS C:\Users\mjjha\Documents\Checkrow&gt; python -u &quot;c:\Users\mjjha\Documents\Checkrow\1.py&quot;
  http://www.google.co.in/imghp?hl=en&amp;tab=wi
  http://maps.google.co.in/maps?hl=en&amp;tab=wl
  https://play.google.com/?hl=en&amp;tab=w8
  http://www.youtube.com/?gl=IN&amp;tab=w1
  https://news.google.com/?tab=wn
  https://mail.google.com/mail/?tab=wm
  https://drive.google.com/?tab=wo
  https://www.google.co.in/intl/en/about/products?tab=wh
  http://www.google.co.in/history/optout?hl=en
  /preferences?hl=en
  https://accounts.google.com/ServiceLogin?hl=en&amp;passive=true&amp;continue=http://www.google.hr/webhp%3Ftbs%3Dsbi:AMhZZiu1U_1mLzDh7oSZSFpNdcot7N84lXExmiJp6LMIJ1NO2PHle7mcBr72CTgX45DTbkkF8yfhvT0GATXTIzgd--ayauOaI-gLvTa-DAeOAodk35Kz6mpzCzl8ly6YYdUbn2S5cCe35BP37ysxT-tSFbvtLPwovJuiNPmunzpk_1j0a88zkXOmb1tn3vfgXnb6IhaucZJIMZztSDOIljOiaSTIzhdQ1aSusETDAu3EMNnoWRaFWqzcUGHzIWuABI9gJkelzjDaV-aK4ilxQJhwGJnzuKNHDbJ4GSX33an2jIfssmwfoWZLFej_1V0Zijr2fuFqULhoAg2lku41nHNxHY1nI0gNU4M2Q&amp;ec=GAZAAQ
  /search?ie=UTF-8&amp;q=Anne+Frank&amp;oi=ddle&amp;ct=236393864&amp;hl=en-GB&amp;si=AC1wQDDagiMg03ncxeOQZbwVe-CJxRCchC-jr2hCPTxjc9wbgOxFdg4PkIAWeA8WhyCLGGzkibRoi5B84SONt2NaUNMtZff0HVDXAtNUKeMfxbgImvSIzyY%3D&amp;sa=X&amp;ved=0ahUKEwj64PWzk8f4AhXqS2wGHdm9C0MQPQgD
  /advanced_search?hl=en-IN&amp;authuser=0
  http://www.google.hr/setprefs?sig=0_S9cBw3JrMA0drw1wRsLp0wK1RFM%3D&amp;hl=hr&amp;source=homepage&amp;sa=X&amp;ved=0ahUKEwj64PWzk8f4AhXqS2wGHdm9C0MQ2ZgBCAU    
  http://www.google.hr/setprefs?sig=0_S9cBw3JrMA0drw1wRsLp0wK1RFM%3D&amp;hl=hi&amp;source=homepage&amp;sa=X&amp;ved=0ahUKEwj64PWzk8f4AhXqS2wGHdm9C0MQ2ZgBCAY    
  http://www.google.hr/setprefs?sig=0_S9cBw3JrMA0drw1wRsLp0wK1RFM%3D&amp;hl=bn&amp;source=homepage&amp;sa=X&amp;ved=0ahUKEwj64PWzk8f4AhXqS2wGHdm9C0MQ2ZgBCAc    
  http://www.google.hr/setprefs?sig=0_S9cBw3JrMA0drw1wRsLp0wK1RFM%3D&amp;hl=te&amp;source=homepage&amp;sa=X&amp;ved=0ahUKEwj64PWzk8f4AhXqS2wGHdm9C0MQ2ZgBCAg    
  http://www.google.hr/setprefs?sig=0_S9cBw3JrMA0drw1wRsLp0wK1RFM%3D&amp;hl=mr&amp;source=homepage&amp;sa=X&amp;ved=0ahUKEwj64PWzk8f4AhXqS2wGHdm9C0MQ2ZgBCAk    
  http://www.google.hr/setprefs?sig=0_S9cBw3JrMA0drw1wRsLp0wK1RFM%3D&amp;hl=ta&amp;source=homepage&amp;sa=X&amp;ved=0ahUKEwj64PWzk8f4AhXqS2wGHdm9C0MQ2ZgBCAo    
  http://www.google.hr/setprefs?sig=0_S9cBw3JrMA0drw1wRsLp0wK1RFM%3D&amp;hl=gu&amp;source=homepage&amp;sa=X&amp;ved=0ahUKEwj64PWzk8f4AhXqS2wGHdm9C0MQ2ZgBCAs    
  http://www.google.hr/setprefs?sig=0_S9cBw3JrMA0drw1wRsLp0wK1RFM%3D&amp;hl=kn&amp;source=homepage&amp;sa=X&amp;ved=0ahUKEwj64PWzk8f4AhXqS2wGHdm9C0MQ2ZgBCAw
  http://www.google.hr/setprefs?sig=0_S9cBw3JrMA0drw1wRsLp0wK1RFM%3D&amp;hl=ml&amp;source=homepage&amp;sa=X&amp;ved=0ahUKEwj64PWzk8f4AhXqS2wGHdm9C0MQ2ZgBCA0    
  http://www.google.hr/setprefs?sig=0_S9cBw3JrMA0drw1wRsLp0wK1RFM%3D&amp;hl=pa&amp;source=homepage&amp;sa=X&amp;ved=0ahUKEwj64PWzk8f4AhXqS2wGHdm9C0MQ2ZgBCA4    
  /intl/en/ads/
  http://www.google.co.in/services/
  /intl/en/about.html
  http://www.google.hr/setprefdomain?prefdom=US&amp;sig=K_5Sxk31MIG7AiUTjwI71yoWFyO_E%3D
  /intl/en/policies/privacy/
  /intl/en/policies/terms/
  </code></pre>
  <p>The content fetched by requests module doesn't contain the full web page I don't know why. I want to scrape information in image ,anchor and h3 tags from the page using beautiful soup but its just not working out.</p>