Site URL displays "Page Not Found The requested URL was not found on this server." but the backend loads fine
Got it working! Not really sure exactly what was causing it, but it seems to have had something to do with my Capistrano setup not craft or Apache.