Would it be rude to ask a famous professor who doesn't know me personally for a recommendation letter if I have published in a high ranking journal?
The benefit of recommendation letters is that they give insight into abilities that are not reflected in the rest of your application. Your grades and your publication are already in your application. A professor who doesn't know anything else about you has nothing to write that will help you.

To directly answer your question: it comes across more naïve/inexperienced than rude, but either way I do not see how it will help your case.

P.S. This answer assumes that the professor is not already familiar with your work before your email. To be more specific, I'm imagining you sending unsolicited emails to a bunch of the biggest names in your field (selected purely because they are "famous"), who have not previously heard of you or your work (as far as you know), asking them to write a recommendation letter for you entirely on the basis of your having published a paper in a journal.

If the professor is already familiar with your work (e.g. has cited it, or has had an email correspondence with you about it), then I agree with this answer that his/her letter can potentially be helpful. Similarly, I think Ben Webster offers some useful advice about attempting this in a more nuanced - and potentially more effective - way.