Wordpress & Mod Rewrite
I found a way to make this work, though I don't know if it is a solution or a workaround...

I made a page called Wordpress, and made the auto-generated CiviCRM page a child page of that page. Therefore the URL of that page is mysite.com/wordpress/civicrm and I suppose because that page exists, WP does not try to rewrite it? For some reason, it works now...