Why did the Apache directive to handle a clean URL get changed? I noticed that the clean URLs implementation has gotten more complicated in Drupal 7. Rather than RewriteRule ^(.*)$ index.php?q=$1 [L,QSA] used in Drupal 6, Drupal 7 .htaccess has simply RewriteRule ^ index.php [L], with a comment that says "Clean URLs are handled in drupal_environment_initialize()".

I read through drupal_environment_initialize() and understand that it's using REQUEST_URI to manually set $_GET['q'], but why the change? I see the comment:

// When clean URLs are enabled, emulate ?q=foo/bar using REQUEST_URI. It is
// not possible to append the query string using mod_rewrite without the B
// flag (this was added in Apache 2.2.8), because mod_rewrite unescapes the
// path before passing it on to PHP. This is a problem when the path contains
// e.g. "&amp;" or "%" that have special meanings in URLs and must be encoded.
$_GET['q'] = request_path();


I've got some idea what this is saying, but can anyone explain in fuller terms the reasons behind the change? If it wasn't done this way before, was something (minorly or seriously) broken in Drupal 6 and prior?
There was something seriously broken, but it only affected a handful of cases due to fringe nature of the issue. Essentially, what happens is roughly (%26 == &amp;):

Request: /lost%26found
Rewrite: /index.php?q=lost&amp;found


The issue here is obvious when you see it this way: you end up with two $_GET variables, q = 'lost' and found which isn't set properly. Since the q variable is truncated, you end up with the wrong page or a 404 error.

Drupal 7 fixes this by simply redirecting to index.php and then parsing the original query string manually itself, bypassing Apache and PHP's parsing of it entirely.