title: Regex over changing conditions in email Python
body: <p>I have employee-client email exchanges that need client message bodies pulled, to be consumed for future sentiment analysis.</p>
  
  <p>These emails were generated using different email applications, so there's not a single regex rule I can use to separate the emails, and they don't all conform to the form used by the email module, so object-style parsing isn't a possibility. Sometimes different email applications were mixed in a single chain, so I can't regex on a specific profile either.</p>
  
  <p>However, these rules have cropped up as reliable:</p>
  
  <p>Email Start:</p>
  
  <ul>
  <li>*@[not acme] wrote:</li>
  <li>[newline]to: *@acme.com</li>
  <li>[newline]to: *'acme support'</li>
  <li>beginning of the chain</li>
  </ul>
  
  <p>Email End:</p>
  
  <ul>
  <li>*@[acme]> wrote:</li>
  <li>[newline]From: *@acme.com</li>
  <li>end of the chain</li>
  </ul>
  
  <p>These can be mixed and matched during the life of a chain. A message can start with a 'wrote' rule, and end with a '[newline]From:' rule, or '*@acme> wrote' rule, etc.</p>
  
  <p>Is there any elegant way to set different start and end conditions for this regex? Ideally, it would lazily stop at the first instance of hitting one of the end rules.</p>
  
  <p>FWIW, I consider myself strictly intermediate with python. Experienced enough to struggle meaningfully through documentation, but not enough to play with the deeper layers of the language.</p>
  
  <p>Example Source Data:</p>
  
  <pre><code>thank you john
  
  
  
  from: noreply@acme.com [mailto:noreply@acme.com] on behalf
  of acme help
  sent: thursday, december 29, 2016 11:28 am
  to: Jane Doe
  subject: re: aha - overtime
  
  
  
  
  
  
  hi Jane,
  
  
  
  it is affected by your payroll schedule. because it is semi-monthly,
  overtime is a tricky thing to calculate, so we have to make sure we do
  it just right! once i turn this setting on, you will be good to go from
  this point on!
  
  
  
  best regards,
  
  
  
  john doe
  
  customer experience team 
  
  
  []
  
  
   &lt;http://portal.mxlogic.com/images/transparent.gif&gt; 
  
  
  
   &lt;http://portal.mxlogic.com/images/transparent.gif&gt; 
  
  ref:_00d15ft7b._50015ypl8b:
  
  Jane_Doe@her_company.com wrote:
  refit will not come up, even after logging on. i have my pass word and user
      name write on a sheet of paper in my wallet, so i know it is correct. it
  looks like it is trying to come up, but all i see is  two arrows going
  in circles..
  
  
  
  Jane
  
  
  
  from: acme support [mailto:help@acme.com] 
  sent: thursday, january 05, 2017 10:42 am
  to: Jane Doe
  subject: [graymail] re: happy new year from acme!
  
  
  
  hello Jane,
  
  
  
  sorry to hear that you're having trouble using acme. can you please
  elaborate on the issue that you're experiencing?
  
  
  
  best regards,
  
  
  
  john
  
  customer experience team 
  
  
  
  
  --------------- original message ---------------
  from: Jane Doe [jane_doe@her_company.com]
  sent: 1/5/2017 11:34 am
  to: support@acme.com
  subject: re: [graymail] happy new year from amce!
  
  our acme app is not working
  </code></pre>
  
  <p>Desired result (in any format, I've stored earlier, simpler regex in lists by leveraging re.findall()):</p>
  
  <pre><code>thank you john
  
  refit will not come up, even after logging on. i have my pass word and user
          name write on a sheet of paper in my wallet, so i know it is correct. it
      looks like it is trying to come up, but all i see is  two arrows going
      in circles..
  
  
  
      Jane
  
  our acme app is not working
  </code></pre>
  
  <p>EDIT:</p>
  
  <p>I was able to parse chat logs earlier using code like this. The source data is currently stored in a pandas dataframe consisting of just a client_id-log pair. My current problem is structured identically, in client_id - email_chain pairs:</p>
  
  <pre><code>for index, row in df_chttext.iterrows(): #for each client-chat item:
      list_cleaned = [] #clear out old list_cleaned
  
      chat = row['chat_log'] #grab chat log
      list_visitor = re.findall('Visitor: .*?&lt;br&gt;', chat) #get list of only visitor messages
  
      if list_visitor: #if there is a list of client messages
          for message in list_visitor: #scrub the message
              scrub = message.replace('Visitor: ','')
              scrub = scrub.replace('&lt;br&gt;','')
              scrub = scrub.replace('&amp;#39;','\'')
              scrub = scrub.replace('&amp;gt;','&gt;')
              scrub = scrub.replace('&amp;lt;','&lt;')
              list_cleaned.append(scrub)
          df_chttext.at[index,'chat_log'] = list_cleaned #replace previous chat with scrubbed chat
      else:
          df_chttext.at[index,'chat_log'] = '' #if no user messages, then leave it empty
  </code></pre>