Why did the Apache directive to handle a clean URL get changed?
There was something seriously broken, but it only affected a handful of cases due to fringe nature of the issue. Essentially, what happens is roughly (%26 == &amp;):

Request: /lost%26found
Rewrite: /index.php?q=lost&amp;found


The issue here is obvious when you see it this way: you end up with two $_GET variables, q = 'lost' and found which isn't set properly. Since the q variable is truncated, you end up with the wrong page or a 404 error.

Drupal 7 fixes this by simply redirecting to index.php and then parsing the original query string manually itself, bypassing Apache and PHP's parsing of it entirely.