How does harvest time matter with tea?
Harvest time can have a large effect on the quality of the tea. First off, whether it was the first harvest of the year or a later one will greatly change the character of the tea. 

The dates can also be used similar to the vintage dates of wine. If you got a tea from a specific region and know the date of the harvest, if you find another tea from the same region and same date, it will have been grown under similar conditions.

Also, since black and oolong teas have been oxidized, the oxidation process will continue to some extent as the tea ages, further changing the flavor. The date can be used to predict freshness, and also to find a tea that suits your tastes better (for instance, I know that I like many black teas better after they have aged for a year or two).