Scraping financial web data
I too would fall back on parsing either the HTML or the entities using regular expressions. My experience is though that this always gets unelegant quickly.


Do you have a somewhat clear idea of the relevant sources? If the
better part of the relevant data comes from a limited number of
pages, you could maintain a list of sources with matching wrappers. 
Then within those relevant documents, I would search for the least
complex most valuable features to extract.


Example

For instance, if you'd be interested in the quarterlies of Alphabet, I would scrape this link. You're smart, you can figure out the next one.

A quick glance learns me that the first hit on revenue(s)  $ returns me the revenue for the quarter. 

So something like this:

(?:revenue[s]?)(?:\s[\w]+\s)(\$[\d]+\.?\d\s[\w]+)


Testing that one the reports on the site seems to work on q1, q2 and q3 while q4 yields the annual revenue. Easy enough to fix.

My experience is that thee patterns hold for a while, and then change. No big deal, just add a couple of tests! Fi: Is the result not empty and is it in a believable range?