Python Web Scrape Write Output to File
Question: I have a basic Python Script which can store the output to a file. This is
file is difficult to parse. Any other way to write scraped data to a file
which can be read easily into Python for analysis ?

    
    
    import requests
    from bs4 import BeautifulSoup as BS
    import json
    data='C:/test.json'
    url="http://sfbay.craigslist.org/search/sby/sss?sort=rel&query=baby" 
    
    r=requests.get(url)
    soup=BS(r.content)
    links=soup.find_all("p")
    #print soup.prettify()
    
    for link in links:
        connections=link.text
        f=open(data,'a')
        f.write(json.dumps(connections,indent=1))
        f.close()
    

Output File contains this: " $25 Sep 5 Porcelain Baby Deer $25 (sunnyvale) pic
household items - by owner "" $7500 Sep 5 GEORGE STECK BABY GRAND PLAYER PIANO
$7500 (morgan hill) map musical instruments - by


Answer: If you want to write it from python to a file, and read it back into python
later, you can use Pickle - [Pickle
Tutorial](https://wiki.python.org/moin/UsingPickle).

Pickle files are in binary and will not be human-readable, if that's important
to you then you could look at yaml, which I'll admit has a bit of a learning
curve, but produces nicely formatted files.

    
    
    import yaml
    
    f = open(filename, 'w')
    f.write( yaml.dump(data) )
    f.close()
    
    ...
    
    
    stream = open(filename, 'r')
    data = yaml.load(stream)
    

