Professor not replying to my email
Should I go directly and talk with the Professor, assuming that he did not notice my email?


Yes.  Well, whether you believe that he read your email or not is not really the point: you didn't get an answer.  You are entitled to one (not necessarily a great or helpful one, but at least an answer in the formal sense).


  Even if I was not able to arrange an appointment with him via email, should I directly knock his door[?]


Yes, exactly.


  [W]ill this be considered rude? 


Students are allowed to visit faculty in their offices, even without an appointment.  Again, what happens when they do this is not guaranteed: maybe you will only get a minute of the professor's time.  (Maybe he won't be there...) But you can show up.  As to whether the professor will consider it rude: not in a way that he is entitled to complain about, no.  Professors are real people, and real people usually want to talk to certain real people more than others, and when a person you don't want to talk to, or don't want to talk to now or today nevertheless appears, are you thrilled about it?  But none of this really matters: if you don't get a response, you keep trying.  That's life.