Trimming unicode whitespace, control characters, and line breaks with regex I'm getting user submitted text from a &lt;textarea&gt;&lt;/textarea&gt;, and I want to edit it a bit before storing it in a database. Here are my 3 regexes, with explanations below them:


/(?:^((\pZ)+|((?!\R)\pC)+)(?1)*)|((?1)$)|(?:((?2)+|(?3)+)(?=(?2)|(?3)))/um


For every line...


Match all leading whitespace and unicode control characters ("WS/CC") (EXCEPT LINE BREAKS).
Match all trailing WS/CC (EXCEPT LINE BREAKS). 
Match all but one WS/CC between non-whitespace characters.

regex101

/(\pZ+)|((?!\R)\pC)/u


Match remaining WS/CC (EXCEPT LINE BREAKS).
regex101

/(^\R+)|(\R+$)|(\R(?=\R{2}))/u


Match all leading line breaks.
Match all trailing line breaks.
Match all but 2 consecutive line breaks.
regex101



This is pretty much how I'm planning on using these regexes in my PHP script

// ...
// [Get user input from $_POST
// check its length, etc.,
// if everything looks O.K., store it in $str]
// ...

// Trim unnecessary whitespace (leave line breaks)
$str = preg_replace("/(?:^((\pZ)+|((?!\R)\pC)+)(?1)*)|((?1)$)|(?:((?2)+|(?3)+)(?=(?2)|(?3)))/um", '', $str);
// Convert remaining whitespace to regular spaces (leave line breaks)
$str = preg_replace("/(\pZ+)|((?!\R)\pC)/u", ' ', $str);
// Trim line breaks
$str = preg_replace("/(^\R+)|(\R+$)|(\R(?=\R{2}))/u", '', $str);

// Sanitize for safe printing between html tags
$str = htmlspecialchars($str, ENT_HTML5, 'UTF-8');

// ...
// [Store $str in DB with prepared statements]
// ...


And here's a test case which I've executed on regex101:



Input
(with some WS/CC in it):



a b  c   
 d    e
fgh​​i  j




k l m n   
op




Output after 1st regex
(still has some WS/CC, but none consecutive, except for line breaks):



a b c
d e
fghi j




k l m n
op




Output after 2nd regex
(all previous WS/CC - except for line breaks - are now regular spaces):



a b c
d e
f g h i j




k l m n
o p




Output after 3rd regex
(max 2 consecutive line breaks, and no whitespace other than non-consecutive regular spaces):
a b c
d e
f g h i j

k l m n
o p


Based on my testing, this process seems to work as intended. I would however be very interested if you can think of any cases in which it would fail to behave as expected.

Also - in addition to possible fail-cases - I'd like to know if there are ways I could make these regexes perform better (faster), while still keeping the functionality as is. This is pretty much the first time I've ever done anything in regex, so I'm assuming there's some room for optimizations.

And if you see something else that is done incorrectly/could be done better, do let me know.

EDIT: Added regex101 links
EDIT2: Also, I've heard there's something called ReDoS, or Regex DoS. If you can tell that these regexes are susceptible to those, and which ones specifically, I'd like to know how to avoid them.
So I've toiled away at this for a while and although I am feeling a little out of my depth I'll submit my approach which employs what I do understand about preg_replace and regex.


preg_replace() allows mixed type pattern and replacement parameters, so one call can execute multiple patterns/replacements.
Speeding up regex can be achieved by minimizing capture groups and alternatives (pipes) as well as using character classes and negated characters classes where appropriate.


I've prioritized accuracy and endeavored to provide the most robust patterns considering all possible character sequences that may be submitted from the textarea.  Then I've attempted to write each pattern with greater efficiency (without impacting accuracy).  Finally, I've tried to reduce the overall length and convolution of the regex patterns.

I want to be clear that I don't have any experience with recursive patterns.  I don't know how/when to use them effectively and so none of my patterns employ them.

This is the sample input that I used which has whitespaces, newlines (single and greater than double), and control characters:

$text="\0\n\n\1\n\r\n\13\27a ab\r\n\t\t\r\r\ncà\1êß€\0\0abcbc   d\n\n\t\r\n\5   e\2\n\3\n\4\n";


This input string was designed to be as difficult as possible to best test my method.

$patt_repl=[
    '/^[\s\pC]+|[\s\pC]+$/u'=&gt;'',  // aka mb_trim() -- remove all leading and trailing whitespace and control characters. DO NOT ADD m FLAG TO PATTERN, THAT WILL DAMAGE THE STRING
    '/[\s\pC]*?(\R)[\s\pC]*?(\R)[\s\pC]*/u'=&gt;"\n\n",  // cleans 2 or more new lines (uninterupted by non-white space characters)
    '/(?!\R)[\h\pC]*\R[\h\pC]*(?!\R)/u'=&gt;"\n",    // cleans single new lines
    '/\n+(*SKIP)(*FAIL)|[\h\pC]+/u'=&gt;' '  // convert 1 or more non-newline white-spaces and control characters to single space
];

$text=htmlspecialchars(preg_replace(array_keys($patt_repl),$patt_repl,$text), ENT_HTML5, 'UTF-8');

echo "This is the post-replace/result string dump:\n";
var_dump($text);


Yes, my method uses 4 separate patterns rather than 3 like the OP.  This is for improved accuracy.  I believe the OP's method had some gaps in accuracy, so that is the justification for the additional pattern.

The output from the above method is:

This is the post-replace/result string dump:
string(28) "a ab

cà êß€ abcbc d

e"


...which is what I believe the output should be.


It trims all leading and trailing whitespace and control characters from the input string.
It reduces two or more newline characters (which may have leading, in-between,  trailing whitespace or control characters) to \n\n.
It trims all leading and trailing horizontal whitespace or control characters from all single newline characters.
It converts 1 or more consecutive horizontal whitespace or control characters to  (a single space).  This, during my development, required me to use the (*SKIP)(*FAIL) technique to disqualify newline characters becausse \h wasn't doing what I expected.  I have read on SO and elsewhere that there can be some quirky/untrustworthy behaviors when dealing with \h and \v while processing certain characters on certain systems -- but I do not possess the necessary wisdom to explain this.


For my own confidence, I wanted to have a way to confirm the existence/absense of whitespace and control characters in the string before and after the method.  I found a very helpful function in the top comment at the php manual's ord() page.

Here is a PHP demo that iterates the string and identifies each and every character to prove effectiveness.

Here are a few relevant regex links that I visited during my research:

https://www.regular-expressions.info/unicode.html
https://stackoverflow.com/questions/3230623/filter-all-types-of-whitespace-in-php
https://stackoverflow.com/questions/5471644/what-are-the-whitespaces-matched-by-s-in-php

On the matter of ReDoS and Catastrophic Backtracking, the threat is minimal (if not non-existent) because of my use of character classes and the structure of my alternatives.  In case you haven't already visited it, here is a good page to read if you want to investigate further.

Finally, I humbly ask that if anyone has a string that breaks my method, please provide it to me so that I can update my answer.  And if any of my assertions are incorrect, please correct me.