Is it rude to call a professor by their last name with no prefix in a non-academic setting?
I wouldn’t find it rude, but I would find it weird.  I can’t think of any situation where you wouldn’t be better off using a first name instead of a last name without a title.  If you’re going to be formal do it right, and if you’re going to be informal use first name.  We’re not on a football team or in the military.

(It’s of course totally fine and normal students talking to each other without the professor present, to just use last name and no title.)