Why doesn't telomerase activity cause DNA to get longer each time a cell undergoes DNA replication?
I will assume that you are referring to humans, though much of the research to elucidate telomerase function was performed in yeast.

The first reason is that only a small subset of somatic cells express telomerase. Most somatic cells are terminally differentiated and mitotically inactive, so they are not called upon to replicate their DNA and divide. And when they do, their telomeres get shorter due to the 3' end replication problem.

The end replication problem arises due to the biochemical limitations of DNA Polymerase. Pol requires a primer, 5' to 3', in order to bind to the template strand and begin to catalyze the elongation reaction. So when DNA Polymerase gets to the 3' end of the template strand, it is unable to catalyze the reaction because it doesn't have a free 3' hydroxyl. When the end RNA primer that was used to catalyze the Okazaki fragment just before the 3' end is removed, there is no place left on the template strand to prime, so there is no 3' hydroxyl to begin the elongation reaction and you are left with a 3' overhang of single-stranded DNA. 

On the 5' end of the Template Strand, even though the polymerase is able to catalyze the reaction all the way to the end, the end of the chromosome is processed and nucleotides are removed to make a single-stranded 3' overhang on this end. The overhangs are necessary to do strand invasion to form the T-Loops of the telomeres. So every time a chromosome in a cell that has inactive telomerase replicates, it gets shorter.

For the cell types that have telomerase actively express, such as lymphocytes, partially-differentiated stem cells, and the precursor cells to sperm, there is a negative feedback loop based on the length of the telomeres, which in turn regulates the activity of telomerase. Telomerase maintains the ends of the chromosomes, especially in these mitotically active cells, but it will only elongate them to a certain point.


  It has long been known that telomere length is regulated by a negative feedback loop (reviewed in Smogorzewska and de Lange, 2004). Long telomeres contain more negative regulators that limit further telomere elongation by telomerase. Because of this, the enzyme preferentially acts on the shortest telomeres in the cell (Hemann et al., 2001; Zhu et al., 1998).
  - Loayza, et. al.; Cell; Volume 117, Issue 3 30 April 2004