title: Reading a specification document (PDF)with paragraphs and tables into a spreadsheet
body: <p>My engineering team is gearing up for a bidding on a public project, where the specifications document is huge (~500 pages). I would like to break it down clause by clause in a spreadsheet and then assign the teams the relevant &quot;portion&quot;. I checked, but PDF document is the only way these specs are provided.</p>
  <p>The idea is to record it such that we can compare it with specifications of previous projects that are recorded in similar manner. I am still a trainee, so am not aware how this process works around different companies, but here in my team, the last project was documented manually in a similar manner.</p>
  <p>The pages are arranged in indexed paragraphs (as 1, 1.1, 1.1.1 etc) with some tables and figures thrown around.</p>
  <p>I hope to get a table like this:</p>
  <div class="s-table-container">
  <table class="s-table">
  <thead>
  <tr>
  <th>Clause No.</th>
  <th>Clause Para</th>
  </tr>
  </thead>
  <tbody>
  <tr>
  <td>1.1.</td>
  <td>Lorem Ipsum</td>
  </tr>
  <tr>
  <td>1.2.</td>
  <td>Lorem Ipsum</td>
  </tr>
  </tbody>
  </table>
  </div>
  <p>I asked around on PM Stackexchange if someone had some idea regarding any software suite, but I don't think there are many.</p>
  <p>So I turned to R hoping that I could solve this maybe by parsing it using pdftools and a regex, and generally, while checking the code, I can get it to run on regex101.com to some extent (randomly selects few paragraphs, but fails when encounters a table) but somehow it does not return the same response when used with R.</p>
  <p>I have no commitment to use R, but it is just that it was easilty available on my work laptop. Willing to try python or any other toolkit as well.</p>
  <p>So far, I have been stuck on getting to make R get a single paragraph.</p>
  <pre><code>library(pdftools)
  library(dplyr)
  library(stringr)
  library(purrr)
  
  setwd(&quot;The work Dir/&quot;)
  specDoc &lt;- pdf_text(&quot;Spec Doc.pdf&quot;) %&gt;% strsplit(split = &quot;\n&quot;)
  specDocChar &lt;- as.character(specDoc)
  
  get_clause &lt;- str_trim(str_extract(specDocChar, &quot;(?:^\n*(?:\\d\\.(?:\\d\\.)*)+)(.+?)$&quot;))
  
  get_clause
  
  </code></pre>
  <p>I tried the <code>lookbehind</code> also, but it seems to not work with flexible starting string lengths.</p>
  <p>At this point I wish to know two things mainly.</p>
  <ol>
  <li>What am I doing incorrectly that I end up having a blank output</li>
  <li>Is there a more efficient way to tackle this particular problem, because after the paragraphs, I am not sure how to manage the tables within the paragraph, and para alone takes a little too much time.</li>
  </ol>
  <p><strong><a href="https://drive.google.com/file/d/1YXc4i9yzDZRv3XKifO3fF5-dLurfB232/view?usp=sharing" rel="nofollow noreferrer">A sample of how the page looks</a></strong></p>
  <p><strong><a href="https://docs.google.com/spreadsheets/d/1y8t2CiedID0XRzGBspvP5zRtvnNVjg167HhLR9_LCCc/edit?usp=sharing" rel="nofollow noreferrer">Expected Output</a></strong></p>