Is it normal for relay to have quiet buzzing noise?
It's not nominal but not uncommon, so it'll depend on your definition of normal. 

Most likely you are not providing enough power to hold the relay in whatever state it's in when powered. If it's close to the correct value, it'll operate, but will rattle or buzz.