"Craft CMS does not support backtracking to this version. Please upload Craft CMS 0 or later."
Turns out I hadn't fully uploaded the app folder when I tried to access it earlier. Tried again, made sure it was finished before renaming and it worked fine.