Web-scraping a list of lawyers I've written a program to get the names along with the titles of some practitioners out of a webpage. The content stored within disorganized HTML elements (at least it seemed to me) and as a result the output becomes messier. However, I've managed to get the output as it should be.

The way I've written the scraper serves its purpose just fine but the scraper itself looks ugly. Is there any way to make the scraper look good and still have it do the same job as it is doing currently?  For example, is there a way to eliminate the exception handlers?

This is how I've written it:

import requests 
from bs4 import BeautifulSoup

url = "https://www.otla.com/index.cfm?&amp;m_firstname=a&amp;seed=566485&amp;fs_match=s&amp;pg=publicdirectory&amp;diraction=SearchResults&amp;memPageNum=1"

def get_names(link):
    res = requests.get(link)
    soup = BeautifulSoup(res.text,'lxml')
    for content in soup.select(".row-fluid"):
        try:
            name = content.select_one(".span5 b").get_text(strip=True)
        except: continue  #let go of blank elements
        else: name

        try:
            title = content.select_one(".span5 b").next_sibling.next_sibling.strip()
        except: continue  #let go of blank elements
        else: title

        print(name,title)

if __name__ == '__main__':
    get_names(url)
Don't use a bare except statement if it's not absolutely necessary. Catch the errors you anticipate and explicitly want to handle.

Also, there's no reason for the duplicated try/except/else statements, and no use, in this case, of the else statements at all. You can collapse everything into a single try/except, since you do the same thing regardless of which line throws the error.

You also have a little code duplication. Don't Repeat Yourself (DRY).

def get_names(link):
    res = requests.get(link)
    soup = BeautifulSoup(res.text,'lxml')
    for content in soup.select(".row-fluid"):
        try:
            element = content.select_one(".span5 b")
            name = element.get_text(strip=True)
            title = element.next_sibling.next_sibling.strip()
            print(name,title)
        except AttributeError:
            continue