Cannot use the alias "/libraries"
For anyone looking in the future, this will solve it.

Add this to your htaccess file:

# Allow the alias "/libraries" to work without breaking actual assets in the libraries directory.
RewriteRule ^libraries$ /index.php [L]


Just before this block:

  # Pass all requests not referring directly to files in the filesystem to
  # index.php.
  RewriteCond %{REQUEST_FILENAME} !-f
  RewriteCond %{REQUEST_FILENAME} !-d
  RewriteCond %{REQUEST_URI} !=/favicon.ico
  RewriteRule ^ index.php [L]


I also had to add an .htaccess file to the libraries directory. Inside of that, I had to do:

DirectoryIndex ../index.php
DirectorySlash off


Now I can create an alias for /libraries and not introduce and endless redirect loop, and physical assets (CKEditor plugins) can still be loaded correctly from this directory.