Python Requests gives different page text than Internet Explorer
Question: Looking at my stackoverflow user profile page:
<http://stackoverflow.com/users/2683104/roberto>

The site indicates I have been a member for 316 days (screenshots at end of
post). If I `view source` in my browser (IE11), I can see this data comes from
a `days-visited` class.

But if I look for this same `days-visited` information using Python Requests,
the data does not appear anywhere. Why?

    
    
    from requests import Session
    from BeautifulSoup import BeautifulSoup
    
    s = Session()
    
    url = 'http://stackoverflow.com/users/2683104/roberto'
    page = s.get(url)
    soup = BeautifulSoup(page.text)
    print soup.prettify() #server response, prettified
    
    # following returns error
    # AttributeError: 'NoneType' object has no attribute 'getText'
    #days_visited = soup.find('span', attrs={'id':'days-visited'}).getText()
    
    s.close()
    

**screenshot**

![screenshot](http://i.stack.imgur.com/vvYoI.png)

**view source** ![view_source](http://i.stack.imgur.com/7uXB6.png)

**python Requests** ![python_requests](http://i.stack.imgur.com/RSMoA.png)


Answer: That field is not visible to your script (or other users). If you want to
scrap that piece of information, you will need to have your script login and
store appropriate cookies.

This is what is seen by users that aren't you:
![Profile](http://i.stack.imgur.com/BM6VA.png)

And the code block they see:

    
    
     <tbody>
                <tr>
                    <th>visits</th>
                    <td>member for</td>
    
                    <td class="cool" title="2013-08-14 15:38:01Z">11 months</td>
                </tr>
                <tr>
                    <th></th>
                    <td>seen</td>
    
                    <td class="supernova" title="2014-08-08 05:26:50Z">
                        <span title="2014-08-08 05:26:50Z" class="relativetime">6 mins ago</span>
                    </td>
                </tr>
            </tbody>
    

Normally, I'd recommend against scraping Stack Overflow for data and use the
[API](https://api.stackexchange.com/docs) instead, but this particular piece
of information isn't returned as part of the
[User](https://api.stackexchange.com/docs/types/user) object.

