Light won't turn off
You'll likely have to do some testing to figure out what's what, since you didn't keep track while disconnecting the old fixture.  Start by opening up the switch box, and taking a look at what's going on in there.

Here are two possible wiring configurations, based on your description. 

These diagrams could be completely incorrect, so don't follow them blindly.




Connecting wires simply based on color, is almost never a good idea. A multimeter, and some basic electrical knowledge will help you figure out what goes where. If you lack either of these two things, you might want to consider contacting a local licensed Electrician.