Does every drink bottle fit into every holder?
The inexpensive bottle cages can be bent to make them a little smaller for a tighter fit. If the type you have is a closed  loop type you can add rubber bands to the bottle for a more secure fit if it is loose. There seems to be a lot more size variation with generic "sport bottles" than the bottles marketed to cyclists.