What happens when nobody is eligible for Chancellor in Secret Hitler?
According to the rules:


  If there are only 5 players left in the game, only the last elected Chancellor is ineligible to be the Chancellor Candidate; the last President may be nominated.


So in the situation you described the last President should have been nominated as Chancellor.