<p>I'm currently working on a parser for our internal log files (generated by log4php, log4net and log4j). So far I have a nice regular expression to parse the logs, except for one annoying bit: Some log messages span multiple lines, which I can't get to match properly. The regex I have now is this:</p>

<pre><code>(?&lt;date&gt;\d{2}/\d{2}/\d{2})\s(?&lt;time&gt;\d{2}):\d{2}:\d{2}),\d{3})\s(?&lt;message&gt;.+)
</code></pre>

<p>The log format (which I use for testing the parser) is this:</p>

<pre><code>07/23/08 14:17:31,321 log 
message
spanning
multiple
lines
07/23/08 14:17:31,321 log message on one line
</code></pre>

<p>When I run the parser right now, I get only the line the log starts on. If I change it to span multiple lines, I get only one result (the whole log file).</p>

<hr>

<p>@samjudson:</p>

<p><em>You need to pass the RegexOptions.Singleline flag in to the regular expression, so that "." matches all characters, not just all characters except new lines (which is the default).</em></p>

<p>I tried that, but then it matches the whole file. I also tried to set the message-group to .+? (non-greedy), but then it matches a single character (which isn't what I'm looking for either).</p>

<p>The problem is that the pattern for the message matches on the date-group as well, so when it doesn't break on a new-line it just goes on and on and on.</p>

<hr>

<p>I use this regex for the message group now. It works, unless there's a pattern IN the log message which is the same as the start of the log message.</p>

<pre><code>(?&lt;message&gt;(.(?!\d{2}/\d{2}/\d{2}\s\d{2}:\d{2}:\d{2},\d{3}\s\[\d{4}\]))+)
</code></pre>

[{'answer_id': 41412, 'author': 'samjudson', 'author_id': 1908, 'author_profile': 'https://Stackoverflow.com/users/1908', 'pm_score': 0, 'selected': False, 'text': '<p>You need to pass the <code>RegexOptions</code>. Singleline flag in to the regular expression, so that "." matches all characters, not just all characters except new lines (which is the default).</p>\n'}, {'answer_id': 41423, 'author': 'David Webb', 'author_id': 3171, 'author_profile': 'https://Stackoverflow.com/users/3171', 'pm_score': 1, 'selected': False, 'text': "<p>The problem you have is that you need to terminate the RegEx pattern so it knows when one message ends and then next starts.</p>\n\n<p>When you were running in default mode the newline was working as an implicit terminator.</p>\n\n<p>The problem is if you go into multiline mode there's no terminator so the pattern will gobble up the whole file.  Non-greedy matches a few characters as possible which will be just one.</p>\n\n<p>Now, if use the date for the next message as the terminator I think your parser will only get every other line.</p>\n\n<p>Is there something else in the file you could to terminate the pattern?</p>\n"}, {'answer_id': 41425, 'author': 'mweerden', 'author_id': 4285, 'author_profile': 'https://Stackoverflow.com/users/4285', 'pm_score': 2, 'selected': False, 'text': '<p>You obviously need that "messages lines" can be distinguished from "log lines"; if you allow the message part to start with date/time after a new line, then there is simply no way to determine what is part of a message and what not. So, instead of using the dot, you need an expression that allows anything that does not include a newline followed by a date and time.</p>\n\n<p>Personally, however, I would not use a regular expression to parse the whole log entry. I prefer using my own loop to iterate over each line and use one simple regular expression to determine whether a line is the start of a new entry or not. Also from the point of readability this would have my preference.</p>\n'}, {'answer_id': 41428, 'author': 'Jeff Hillman', 'author_id': 3950, 'author_profile': 'https://Stackoverflow.com/users/3950', 'pm_score': 3, 'selected': True, 'text': '<p>This will only work if the log message doesn\'t contain a date at the beginning of the line, but you could try adding a negative look-ahead assertion for a date in the "message" group:</p>\n\n<pre><code>(?&lt;date&gt;\\d{2}/\\d{2}/\\d{2})\\s(?&lt;time&gt;\\d{2}:\\d{2}:\\d{2},\\d{3})\\s(?&lt;message&gt;(.(?!^\\d{2}/\\d{2}/\n\\d{2}))+)\n</code></pre>\n\n<p>Note that this requires the use of the RegexOptions.MultiLine flag.</p>\n'}, {'answer_id': 41551, 'author': 'Daren Thomas', 'author_id': 2260, 'author_profile': 'https://Stackoverflow.com/users/2260', 'pm_score': 0, 'selected': False, 'text': '<p>You might find it a lot easier to parse the file with a proper parser generator - ANTLR can generate one in C#... Context Free parsers only seem hard until you "get" them - after that, they are much simpler and friendlier to use than Regular Expressions...</p>\n'}]
['https://Stackoverflow.com/questions/41407', 'https://Stackoverflow.com', 'https://Stackoverflow.com/users/909/']