Webcrawler for a car forum
When we are discussing performance of a particular piece of code, it's important to recognize bottlenecks and major contributors to the runtime of the program.

In your particular case, even though you've applied some optimizations like SoupStrainer speed-up for HTML parsing, the synchronous nature of the script is the biggest problem by far. The script is processing pages one by one, not getting to the next page until the processing for the current page is finished.

Switching to an asynchronous approach would be the natural next step in your optimizations. Look into using third-party frameworks like Scrapy or, if you are adventurous, things like asyncio or grequests. 



You could apply one more optimization to your current script which should help you optimize the "crawling/scraping" part - instead of using requests.get(), initialize session = requests.Session() and use session.get() to make requests (documentation). This would allow the underlying TCP connection to be re-used for subsequent requests resulting in a performance increase.