Can a letter from a professor not from my university but a high-ranking university carry extra weight?
Getting a good letter is more important than getting a letter from a professor at University B. So if the professor in question only knows you through your attendance of his course, he most likely won't be able to speak to your research potential. As that is the most important aspect of an application for graduate school, this would not work in your favour. (The points in this answer are excellent guidelines for who to ask for reference letters.)

If he is otherwise qualified to write a good letter for you, then the fact that he is from University B might work slightly in your favour, as admission committees tend to give greater weight to letters of recommendation from researchers that they know.