Complex HTMLs Data Extraction with Python Does anybody know a way of extracting data with python from more convoluted website structures? For example, I'm trying to extract data from the players in the ATP profiles, but it's just so complicated I quit. I think they're pulling data from some database in the script and I suspect that even if I tried I wouldn't be able to get it.

I then started using a specialized software called ParseHub, which pulls the data somewhat visually. It's a pretty good software, but they make it slow on purpose just so you buy it, and it is particularly not cheap.
I ended up using BeautifulSoup to do the job. The final code is not that clean - as I'm kind of a newbie - but it does what I wanted. You can find the source code and the dataset I've extracted so far in this repo. You can also check out an article I did on this in my website: fanaro.com.br.