A web crawler in Python
Some of the general things I would work on:

split the code into separate modules logically. Currently, you have all the code mixed up in single file - you have argument parsing, database interactions, web-scraping code blocks in one place
consistent indentation. Use 4 spaces for indentation
variable naming. Use descriptive variable names. Variable names like l, i or j are not meaningful and raise questions when reading the code

Code Style

avoid handling broad exceptions with a bare except
remove unused imports and re-group them based on PEP8 recommendations
make sure to properly use whitespaces in expressions in statements
put the main execution logic of the program into the if __name__ == '__main__':
you don't need that pass in the get_list_of_urls() function
depth = depth + 1 could be shortened to depth += 1

Performance

since you are requesting the pages from the same host multiple times, consider switching to requests making use of a single session instance which allows to re-use an underlying TCP connection making subsequent requests to the same host faster
importing SoupStrainer was actually a good idea. You can use it and scope the parsing to only the desired parts of the HTML
since you are using regular expressions checks here and there, consider pre-compiling them and using the compiled patterns for searching and matching