title: R - does failed RegEx pattern matching originate in file conversion or use of tm package?
body: <p>As a relative novice in R and programming, my first ever question in this forum is about regex pattern matching, specifically line breaks. First some background. I am trying to perform some preprocessing on a corpus of texts using R before processing them further on the NLP platform GATE. I convert the original pdf files to text as follows (the text files, unfortunately, go into the same folder):</p>
  
  <pre><code>dest &lt;- "./MyFolderWithPDFfiles"
  myfiles &lt;- list.files(path = dest, pattern = "pdf", full.names = TRUE)
  lapply(myfiles, function(i) system(paste('"C:/Program Files (x86)/xpdfbin-win-3.04/bin64/pdftotext.exe"', paste0('"', i, '"')), wait = FALSE))
  </code></pre>
  
  <p>Then, having loaded the <code>tm</code> package and physically(!) moved the text files to another folder, I create a corpus:</p>
  
  <pre><code>TextFiles &lt;- "./MyFolderWithTXTfiles"
  EU &lt;- Corpus(DirSource(TextFiles))
  </code></pre>
  
  <p>I then want to perform a series of custom transformations to clean the texts. I succeeded to replace a simple string as follows: </p>
  
  <pre><code>ReplaceText &lt;- content_transformer(function(x, from, to) gsub(from, to, x, perl=T))
  EU2 &lt;- tm_map(EU, ReplaceText, "Table of contents", "TOC")
  </code></pre>
  
  <p>However, a pattern that is a 1-3 digit page number followed by two line breaks and a page break is causing me problems. I want to replace it with a blank space:</p>
  
  <pre><code>EU2 &lt;- tm_map(EU, ReplaceText, "[0-9]{1,3}\n\n\f", " ")
  </code></pre>
  
  <p>The <code>([0-9]{1,3})</code> and <code>\f</code> alone match. The line breaks don't. If I copy text from one of the original .txt files into the RegExr online tool and test the expression <code>"[0-9]{1,3}\n\n\f"</code>, it matches. So the line breaks do exist in the original .txt file.</p>
  
  <p>But when I view one of the .txt files as read into the EU corpus in R, there appear to be no line breaks even though the lines are obviously breaking before the margin, e.g.</p>
  
  <pre><code>[3] "PROGRESS TOWARDS ACCESSION"
  [4] "1"
  [5] ""
  [6] "\fTable of contents"
  </code></pre>
  
  <p>Seeing this, I tried other patterns, e.g. to detect one or more blank space <code>("[0-9]{1,3}\s*\f")</code>, but no patterns worked.</p>
  
  <p>So my questions are:</p>
  
  <ol>
  <li>Am I converting and reading the files into R correctly? If so, what has happened to the line breaks? 
  <ol start="2">
  <li>If no line breaks is normal, how can I pattern match the character on line 5? Is that not a blank
  space? </li>
  <li>(A tangential concern:) When converting the pdf files, is there code that will put them directly in a new folder?</li>
  <li>Apologies for extending this, but how can one print or inspect only a few lines of the text object? The <code>tm</code> commands and <code>head(EU)</code> print the entire object, each a very long text.</li>
  </ol></li>
  </ol>
  
  <p>I know my problem(s) must appear simple and perhaps stupid, but one has to start somewhere and extensive searching has not revealed a source that explains comprehensively how to use RegExes to modify text objects in R. I am so frustrated and hope someone here will take pity and can help me.</p>
  
  <p>Thanks for any advice you can offer.
  Brigitte</p>
  
  <p>p.s. I think it's not possible to upload attachments in this forum, therefore, here is a link to one of the original PDF documents: <a href="http://ec.europa.eu/enlargement/archives/pdf/key_documents/1998/czech_en.pdf" rel="nofollow">http://ec.europa.eu/enlargement/archives/pdf/key_documents/1998/czech_en.pdf</a><br>
  Because the doc is long, I created a snippet of the first 3 pages of the TXT doc, read it into the R corpus ('EU') and printed it to the console and this is it:  </p>
  
  <pre><code>dput(EU[[2]])
  structure(list(content = c("REGULAR REPORT", "FROM THE COMMISSION ON", 
  "CZECH REPUBLIC'S", "PROGRESS TOWARDS ACCESSION ***********************", 
  "1", "", "\fTable of contents", "A. Introduction", "a) Preface The Context of the Progress Report", 
  "b) Relations between the European Union and the Czech Republic The enhanced Pre-Accession Strategy Recent developments in bilateral relations", 
  "B. Criteria for membership", "1. Political criteria", "1.1. Democracy and the Rule of Law Parliament The Executive The judicial system Anti-Corruption measures", 
  "1.2. Human Rights and the Protection of Minorities Civil and Political Rights Economic, Social and Cultural Rights Minority Rights and the Protection of Minorities", 
  "1.3. General evaluation", "2. Economic criteria", "2.1. Introduction 2.2. Economic developments since the Commission published its Opinion", 
  "Macroeconomic developments Structural reforms 2.3. Assessment in terms of the Copenhagen criteria The existence of a functioning market economy The capacity to cope with competitive pressure and market forces 2.4. General evaluation", 
  "3. Ability to assume the obligations of Membership", "3.1. Internal Market without frontiers General framework The Four Freedoms Competition", 
  "3.2. Innovation Information Society Education, Training and Youth Research and Technological Development Telecommunications Audio-visual", 
  "3.3. Economic and Fiscal Affairs Economic and Monetary Union", 
  "2", "", "\fTaxation Statistics "), meta = structure(list(author = character(0), 
      datetimestamp = structure(list(sec = 50.1142621040344, min = 33L, 
          hour = 15L, mday = 3L, mon = 10L, year = 114L, wday = 1L, 
          yday = 306L, isdst = 0L), .Names = c("sec", "min", "hour", 
      "mday", "mon", "year", "wday", "yday", "isdst"), class = c("POSIXlt", 
      "POSIXt"), tzone = "GMT"), description = character(0), heading = character(0), 
      id = "CZ1998ProgressSnippet.txt", language = "en", origin = character(0)), .Names = c("author", 
  "datetimestamp", "description", "heading", "id", "language", 
  "origin"), class = "TextDocumentMeta")), .Names = c("content", 
  "meta"), class = c("PlainTextDocument", "TextDocument"))
  </code></pre>