File is not creating in python when unicode present in the file name
Question: I need to store HTML file as text files. In the same name of web tittle.
something went wrong with my code so that it is not creating file in side the
directory. I have directory permission to write. I am using Ubuntu 12.04LTS

Directory `/home/user1/` File name print name = Mathrubhumi Sports -
ശ്രീനിക്ക് പച്ചക്കൊടി

The file name contain Unicode values

    
    
      import os
        from urllib import urlopen
        from bs4 import BeautifulSoup
        url= "http://www.mathrubhumi.com/sports/story.php?id=397111"
        raw = urlopen(url).read()
        soup = BeautifulSoup(raw,'lxml')
        texts = soup.findAll(text=True)
        name = soup.title.text
        name= name+'.txt'
        def contains_unicode(text):
            try:
                str(text)
            except:
                return True
            return False
    
        result = ''.join((text for text in texts if contains_unicode(text)))
    
        # Output to a file
        with open(os.path.join('/home/user1/textfiles',name,'w') as out:
            out.write(result)
    

Please help me to debug it


Answer: I tried this and it worked, it created a file called Mathrub*.txt with some
text in it in the current directory.

    
    
    import codecs
    import os
    from urllib import urlopen
    from bs4 import BeautifulSoup
    url= "http://www.mathrubhumi.com/sports/story.php?id=397111"
    raw = urlopen(url).read()
    soup = BeautifulSoup(raw,'lxml')
    texts = soup.findAll(text=True)
    name = soup.title.string
    name= name+'.txt'
    def contains_unicode(text):
        try:
            str(text)
        except:
            return True
        return False
    
    result = ''.join((text for text in texts if contains_unicode(text)))
    # Output to a file
    with codecs.open(name,'w',encoding="utf-8") as out:
        out.write(result)
    

Before adding the codecs part, it complained loudly about trying to write some
characters that it did not know how to interpret.

