Writing to a file and getting weird indentation
<p>I have the following snippet of code, which takes a url opens it, parses out JUST the text and then searches for widgets. The way it detects widgets is by looking for the word <code>widget1</code> and then <code>endwidget</code>, which denotes the end of the widget. </p>

<p>Basically the code writes all lines of text to a file as soon as it finds the word <code>widget1</code> and ends when it reads <code>endwidget</code>. However, my code is indenting all lines after the first <code>widget1</code> line.</p>

<p>This is my output</p>

<pre><code>widget1 this is a really cool widget
       it does x, y and z 
       and also a, b and c
       endwidget
</code></pre>

<p>What I want is:</p>

<pre><code>widget1 this is a really cool widget
it does x, y and z 
and also a, b and c
endwidget
</code></pre>

<p>Why am I getting this indentation? This is my code...</p>

<pre><code> for url in urls:
        page = mech.open(url)
        html = page.read()
        soup = BeautifulSoup(html)
        text= soup.prettify()
        texts = soup.findAll(text=True) 

        def visible(element):
            if element.parent.name in ['style', 'script', '[document]', 'head', 'title']: 
            # If the parent of your element is any of those ignore it

                return False

            elif re.match('&lt;!--.*--&gt;', str(element)):
            # If the element matches an html tag, ignore it

                return False

            else:
            # Otherwise, return True as these are the elements we need

              return True

        visible_texts = filter(visible, texts)

        inwidget=0
        # open a file for write
        for line in visible_texts:
        # if line doesn't contain .widget1 then ignore it
            if ".widget1" in line and inwidget==0:
                match = re.search(r'\.widget1 (\w+)', line)
                line = line.split (".widget1")[1]   
                # make the next word after .widget1 the name of the file
                filename = "%s" % match.group(1) + ".txt"
                textfile = open (filename, 'w+b')
                textfile.write("source:" + url + "\n\n")
                textfile.write(".widget1" + line)
                inwidget = 1
            elif inwidget == 1 and ".endwidget" not in line:
                print line
                textfile.write(line)
            elif ".endwidget" in line and inwidget == 1:
                textfile.write(line)
                inwidget= 0
            else:
                pass
</code></pre>
