title: Creating a new regex based on the returned results and rules of a previous regex | Indexing a regex and seeing how the regex has matched a substring
body: <p>I am particularly looking at R, Perl, and shell. But any other programming language would be fine too.</p>
  
  <p><strong>QUESTION</strong></p>
  
  <p>Is there a way to visually or programmatically inspect and index a matched string based on the regex? This is intended for referencing back to the first regex and its results inside of a second regex, so as to be able to modify a part of the matched string and write new rules for that particular part.</p>
  
  <p><a href="https://regex101.com" rel="noreferrer">https://regex101.com</a> does visualize how a certain string matches the regular expression. But it is far from perfect and is not efficient for my huge dataset.</p>
  
  <p><strong>PROBLEM</strong></p>
  
  <p>I have around 12000 matched strings (DNA sequences) for my first regex, and I want to process these strings and based on some strict rules find some other strings in a second file that go well together with those 12000 matches based on those strict rules.</p>
  
  <p><strong>SIMPLIFIED EXAMPLE</strong></p>
  
  <p>This is my <strong>first regex</strong> (a simplified, shorter version of my original regex) that runs through my <strong>first text file</strong>.</p>
  
  <blockquote>
  <pre><code>[ACGT]{1,12000}(AAC)[AG]{2,5}[ACGT]{2,5}(CTGTGTA)
  </code></pre>
  </blockquote>
  
  <p>Let's suppose that it finds the following three sub-strings in my large text file:</p>
  
  <pre><code>1. AAACCCGTGTAATAACAGACGTACTGTGTA
  2. TTTTTTTGCGACCGAGAAACGGTTCTGTGTA
  3. TAACAAGGACCCTGTGTA
  </code></pre>
  
  <p>Now I have a <strong>second file</strong> which includes a very large string. From this second file, I am only interested in extracting those <strong>sub-strings that match a new (second) regex which itself is dependent on my first regex in few sections</strong>. <em>Therefore, this second regex has to take into account the substrings matched in the first file and look at how they have matched to the first regex!</em></p>
  
  <p>Allow me, for the sake of simplicity, index my first regex for better illustration in this way:</p>
  
  <pre><code>first.regex.p1 = [ACGT]{1,12000}
  first.regex.p2 = (AAC)
  first.regex.p3 = [AG]{2,5}
  first.regex.p4 = [ACGT]{2,5}
  first.regex.p5 = (CTGTGTA)
  </code></pre>
  
  <p>Now my <strong>second (new) regex</strong> which will search the <strong>second text file</strong> and will be dependent on the results of the first regex (and how the substrings returned from the first file have matched the first regex) will be defined in the following way:</p>
  
  <blockquote>
  <pre><code>second.regex = (CTAAA)[AC]{5,100}(TTTGGG){**rule1**} (CTT)[AG]{10,5000}{**rule2**}
  </code></pre>
  </blockquote>
  
  <p>In here <strong>rule1</strong> and <strong>rule2</strong> are dependent on the matches coming from the first regex on the first file. Hence;</p>
  
  <pre><code>rule1 = look at the matched strings from file1 and complement the pattern of first.regex.p3 that is found in the matched substring from file1 (the complement should of course have the same length)
  rule2 = look at the matched strings from file1 and complement the pattern of first.regex.p4 that is found in the matched substring from file1 (the complement should of course have the same length)
  </code></pre>
  
  <p>You can see that second regex has sections that belong to itself (i.e. they are independent of any other file/regex), but it also has sections that are dependent on the results of the first file and the rules of the first regex and how each sub-string in the first file has matched that first regex!</p>
  
  <p>Now again for the sake of simplicity, I use the third matched substring from file1 (because it is shorter than the other two) to show you how a possible match from the second file looks like and how it satisfies the second regex:</p>
  
  <p>This is what we had from our first regex run through the first file:</p>
  
  <pre><code>3. TAACAAGGACCCTGTGTA
  </code></pre>
  
  <p>So in this match, we see that:</p>
  
  <pre><code>T has matched first.regex.p1
  AAC has matched first.regex.p2
  AAGGA has matched first.regex.p3
  CC first.regex.p4
  CTGTGTA has matched first.regex.p5
  </code></pre>
  
  <p>Now in our second regex for the second file we see that when looking for a substring that matches the second regex, we are dependent on the results coming from the first file (which match the first regex). Particularly we need to look at the matched substrings and complement the parts that matched first.regex.p3 and first.regex.p4 (<strong>rule1</strong> and <strong>rule2</strong> from second.regex).</p>
  
  <pre><code>complement means:
  A will be substituted by T
  T -&gt; A
  G -&gt; C
  C -&gt; G
  </code></pre>
  
  <p>So if you have TAAA, the complement will be ATTT.</p>
  
  <p>Therefore, going back to this example:</p>
  
  <ol start="3">
  <li>TAACAAGGACCCTGTGTA</li>
  </ol>
  
  <p>We need to complement the following to satisfy the requirements of the second regex:</p>
  
  <pre><code>AAGGA has matched first.regex.p3
  CC first.regex.p4
  </code></pre>
  
  <p>And complements are:</p>
  
  <pre><code>TTCCT (based on rule1)
  GG (based on rule2)
  </code></pre>
  
  <p>So an example of a substring that matches second.regex is this:</p>
  
  <p>CTAAAACACCTTTGGG<strong>TTCCT</strong>CTTAAAAAAAAAGGGGGAGAGAGAAGAAAAAAAGAGAG<strong>GG</strong></p>
  
  <p>This is only one example! But in my case I have 12000 matched substrings!! I cannot figure out how to even approach this problem. I have tried writing pure regex but I have completely failed to implement anything that properly follows this logic.. Perhaps I shouldn't be even using regex?</p>
  
  <p>Is it possible to do this entirely with regex? Or should I look at another approach? Is it possible to index a regex and in the second regex reference back to the first regex and force the regex to consider the matched substrings as returned by first regex?</p>