Can I disable the control panel?
Good call Brad! Not sure why that didn't occur to me :)

Here's how that's done: 

# Kill admin access
RewriteCond %{REQUEST_URI} ^(.*)/admin
RewriteRule ^(.*)$ - [F,L]