Web scraper for a webpage article I am a beginner in Python and have just coded a simple web scraper for a webpage article, output to a text file, using BeautifulSoup and List.

The code is working fine, but I'm wondering if anybody would know a more efficient way to achieve the same.

import requests
page = requests.get('https://www.msn.com/en-sg/money/topstories/10-top-stocks-of-2017/ar-BBGgEyA?li=AA54rX&amp;ocid=spartandhp')

# 2. Parsing the page using BeautifulSoup
import pandas as pd
from bs4 import BeautifulSoup
soup = BeautifulSoup(page.content, 'html.parser')

# 3. Write the context to a text file
all_p_tags = soup.findAll('p') # Put all &lt;p&gt; and their text into a list
number_of_tags = len(all_p_tags) # No of &lt;p&gt;?

x=0
with open('filename.txt', mode='wt', encoding='utf-8') as file:
    title = soup.find('h1').text.strip() # Write the &lt;header&gt;
    file.write(title)
    file.write('\n')
    for x in range(number_of_tags):
        word = all_p_tags[x].get_text() # Write the content by referencing each item in the list
        file.write(word)
        file.write('\n')
    file.close()
There are at least three things that may help to make the code more efficient:

switch to lxml instead of html.parser (requires lxml to be installed)
use a SoupStrainer to parse only the relevant part of the document
you can switch to http instead of https. While this would bring the security aspect down, you would avoid overhead of SSL handshaking, encryption etc - I've noticed the execution time difference locally, try it out

Improved code:
import requests
from bs4 import BeautifulSoup, SoupStrainer

page = requests.get('http://www.msn.com/en-sg/money/topstories/10-top-stocks-of-2017/ar-BBGgEyA?li=AA54rX&amp;ocid=spartandhp')

parse_only = SoupStrainer(&quot;body&quot;)
soup = BeautifulSoup(page.content, 'lxml', parse_only=parse_only) 

with open('filename.txt', mode='wt', encoding='utf-8') as file:
    title = soup.find('h1').text.strip()
    file.write(title + ' \n')

    for p_tag in soup.select('p') :
        file.write(p_tag.get_text() + '\n')

Note that I've also removed the unused variables and imports.
Btw, if it weren't for the title, we could've pinpointed SoupStrainer to p elements only - might've improved performance even more.