Will people judge me negatively for skipping department seminars?
I would say that you should always go to seminar, unless you have some very compelling reason not to go (you are away, you are working on an experiment, you are trying to finish writing your thesis, etc.). 

There are four reasons:


Scientific courtesy. To travel somewhere and give a talk to the 10 people who show up (5 of whom you already knew) is really irritating. 
Good or bad -- you learn something about presentation. Even if you say "wow, I should never do that in a talk" your hour has been well spent.
You get perspective. You never know when something that someone says will make you see your own work in a different context. 
The speaker may someday be interviewing you for a job. It's better to be able to say "I heard your seminar" than "Oh, sorry, I missed your seminar when you visited."