Accessing website with urllib returns error, retrieving information from Results Page
Question: Hello I created a code in python so I could access a reverse phone lookup site
and determine if a phone is a cell phone or land line. The website I am using
is whitepages, whose results page will only include the phrase "VoIP" if the
phone is a land line (which I have determined after looking at many results).
However, I am getting an error at the website accessing stage. So far my code
looks like:

    
    
    import urllib
    
    def Phone_Checker(number):
         url = 'http://www.whitepages.com/reverse_phone'
         enter = {'e.g. 206-867-5309': number}
         door= urllib.parse.urlencode(enter)
         open=door.encode('UTF-8')
         fight= urllib.request.urlopen(url, open)
         d = fight.read()
         v="VoIP"
         vv=v.encode("UTF-8")
         if vv in d:  #if VoIP  it is landline
              return False
         else:
              return True
    

I changed my strings into bytes because it was required for my variable "open"
to be in bytes for urlopen. In a version of the code I made to access a
different site it required a few other string conversion into bytes but I
cannot quite remember which information required this conversion (just a heads
up if the code after introducing the variable fight looks incorrect because I
have not been able to debug the code which follows because of my difficulty
with my urlopen. Whenever I run my code I receive this error

    
    
    File "C:\Users\aa364\Anaconda3\lib\urllib\request.py", line 589, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
    
    HTTPError: Requested Range Not Satisfiable
    

I was wondering how I could circumvent this error and if there is any possible
alternative to creating a program to verify if a phone is mobile or a landline
for DOMESTIC (USA) phone numbers. Thank you in advance!


Answer: Based on the stuff I'm reading and experimenting with to try to find an answer
on this, I think this is likely whitepages' doing. I have 3 reasons:

  1. the error seems to be a result of whitepages only accepting requests from certain browsers ('User-Agents')
  2. Upon changing the 'User-Agent' I get kicked to robots.txt (which is basically a response meaning "don't automate this")
  3. Both of these things are likely the result of whitepages having a paid/premium-access API: obviously, they'll do whatever they can to stop people from accessing their information for free if they're trying to charge for it

So, I think the answer in this case is, unfortunately, find another
phonenumber lookup.

