How to promote garden drainage?
What you're trying to do is called a French drain system. If you google for French drain you'll find a ton of info and useful diagrams.

Coarse gravel is typically the base. This is then topped with finer gravel and landscape cloth so as to help prevent clogging. If you use sand too, you definitely want to use a cloth to keep the sand from draining out.