How long should I let my new sourdough starter ferment until I can use it?
To make a sourdough starter work you have to create an environment where tasty yeast and bacteria will succeed and harmful or gross microbes will be either crowded out or unable to tolerate the acidity.

This environment is created by continually feeding the starter. That is, adding a consistent ratio of fresh starch to consume.

I read an analogy, perhaps Peter Reinhart's, that feeding a starter is like mowing a lawn. The big weeds won't get the upper hand if the lawn is kept consistently mowed.

Your sourdough recipe will have a ratio of starter to feed for each feeding. If you choose to not discard any starter then Everytime you feed you will have to use more and more flour to maintain the ratio. Depending on the recipe this could double everyday. It's not sustainable.

It sounds like you aren't feeding the sourdough at all. Depending on the temperature it will take only a short time, perhaps a day or two, before nasty microbes get the upper hand and your starter will rot. Believe me, this is an unpleasant experience.

If you are determined to not throw away starter you can collect the discarded portion in the fridge where the fermentation will be slowed. This can be used as filler in other baked goods but will not provide much sourdough flavor or lift until the starter is ready.

At first I hated throwing away starter but I realized I was costing myself a great deal of hassle to save pennies worth of flour.

When feeding a new starter, the amount of time depends on many variables including the feeding ratio and schedule, floor, present microbes, temperature, and desired pungency. Most recipes seem to call for around ten days and that matches my personal experience.