Taxonomy and URL aliases
You will want to use the pathauto module. This will also required for you to install the token module.
Once you have those two modules install you will want to head over to the pathauto settings path at admin/build/path/pathauto.  You will see there's a field-set for Taxonomy. There is also replacements patterns which contains various different tokens you can use to generate your path.
In your case the pattern you want to use looks like this:
category/[catpath-raw]

This will automatically generate the path for those taxonomy terms based of these categories.
I assume most of this is for 6.x but the same concept applies for 7.x