Implementing lock within a Python script I've written a script in Python using multiprocessing to handle multiple process at the same time and make the scraping process faster. I've used locking within it to prevent two processes from changing its internal state. As I'm very new to implement locking within multiprocessing, I suppose there is room for improvement.

What the scraper does is collect the name, address and phone number of every coffee shop traversing multiple pages from Yellowpage listings.

import requests 
from lxml.html import fromstring
from multiprocessing import Process, Lock

link = "https://www.yellowpages.com/search?search_terms=coffee&amp;geo_location_terms=Los%20Angeles%2C%20CA&amp;page={}"
itemstorage = []

def get_info(url,lock,itemstorage):
    response = requests.get(url).text
    tree = fromstring(response)
    for title in tree.cssselect("div.info"):
        name = title.cssselect("a.business-name span")[0].text
        try:
            street = title.cssselect("span.street-address")[0].text
        except IndexError: street = ""
        try:
            phone = title.cssselect("div[class^=phones]")[0].text
        except IndexError: phone = ""
        itemstorage.extend([name, street, phone])
    return printer(lock,itemstorage)

def printer(lock,data): 
    lock.acquire()
    try:
        print(data)
    finally:
        lock.release()

if __name__ == '__main__':
    lock = Lock()
    for i in [link.format(page) for page in range(1,15)]:
        p = Process(target=get_info, args=(i,lock,itemstorage))
        p.start()
General Feedback

This code is fairly straight-forward and easy to read. There are only three functions and none of them extend beyond thirteen lines. If you really wanted more atomic functions you could abstract some functionality from get_info, for instance the code that parses each listing. 

The name link doesn’t feel as appropriate for a string literal that represents a URL as something like url_template. Also see the related section below about naming constants. 

The description was somewhat vague and didn’t specify whether each listing should correspond to an individual list within the returned list but if so, one could use itemstorage.append() instead of itemstorage.extend().

Suggestions

While it isn’t a requirement, it is recommended that each function have a docstring. 


  All modules should normally have docstrings, and all functions and classes exported by a module should also have docstrings. Public methods (including the __init__ constructor) should also have docstrings.1


Additionally, while constants aren’t really different than variables in python, idiomatically uppercase letters are used for the naming constants in python as well as many other languages. As was mentioned above, url_template feels more appropriate for the string currently named link so it may improve readability to use uppercase letters to signify that is a constant: URL_TEMPLATE.

1https://www.python.org/dev/peps/pep-0257/