title: R: web scraping yahoo.finance after 2019 change
body: <p>I have been happily web scraping yahoo.finance pages for a long time using code largely borrowed from other stackoverflow answers and it has worked great, however in the last few weeks Yahoo has changed their tables to be collapsible/expandable tables. This has broken the code, and despite my best efforts for a few days I can't fix the bug.</p>
  
  <p>Here is an example of the code that others have used for years (which is then parsed and processed in different ways by different people).</p>
  
  <pre><code>library(rvest)
  library(tidyverse)
  
  # Create a URL string
  myURL &lt;- "https://finance.yahoo.com/quote/AAPL/financials?p=AAPL"
  
  # Create a dataframe called df to hold this income statement called df
  df &lt;- myURL %&gt;% 
    read_html() %&gt;% 
    html_table(header = TRUE) %&gt;% 
    map_df(bind_cols) %&gt;% 
    as_tibble()
  </code></pre>
  
  <p>Can anyone help?</p>
  
  <hr>
  
  <p>EDIT FOR MORE CLARITY:</p>
  
  <p>If you run the above then view df you get </p>
  
  <pre><code># A tibble: 0 x 0
  </code></pre>
  
  <p>For an example of the expected outcome, we can try another page yahoo hasn't changed such as the following:</p>
  
  <pre><code> # Create a URL string
  myURL2 &lt;-  "https://finance.yahoo.com/quote/AAPL/key-statistics?p=AAPL"
  
  df2 &lt;- myURL2 %&gt;% 
    read_html() %&gt;% 
    html_table(header = FALSE) %&gt;% 
    map_df(bind_cols) %&gt;% 
    as_tibble()
  </code></pre>
  
  <p>If you view df2 you get a tibble of 59 observations of two variables being the main table on that page, beginning with </p>
  
  <p>Market Cap (intraday)5  [value here]
  Enterprise value 3      [value here]
  And so on...</p>