What does it take to get a worthwhile recommendation letter
I mean, is it OK to ask a professor write a recommendation letter for you only based on your behavior in his/her class?


This is fine for one letter, especially if you did really well in their class, went to office hours, got to know them, etc. As you say, your main letters should come from research advisors or people who know you better than just one class.