What happens when I draw cards if I've built the Library and the Observatory?
Q: What if a player has both Library
and Observatory in his city? How do
their effects combine?
A: If this
player draws cards, he draws three and
keeps two.

(Source)
That's straight from the horse's mouth, so to speak.