Does it matter what kind of sugar is used in baking?
Yes!

Sugar is often used as a "wet" ingredient in baking.  That means it needs to be dissolved in the water in order to prevent too much gluten from being produced (making the result fluffy/flaky, and not chewy).  Different sugars hold different amounts of moisture (for example, brown sugar holds more than white) and using sugar with crystals that are too large (or too small) will make the texture come out completely wrong.  Too small and your cake will be rubber; too large and your biscuit will fall apart.

Also, if you're using the sugar for creaming (with fat), you generally need to use a coarser sugar.  Superfine sugar will dissolve too quickly and won't allow enough air to be incorporated.  Confectioner's sugar is good for creaming but has completely different properties from crystal sugar - it's been "processed" and has cornstarch added, so you can't just substitute in equal quantities.

Of course, it goes without saying that the taste is different as well.  But even more important than the taste are the solubility characteristics which, as explained above, will have noticeable effects on other parts of the recipe.

It does depend what you're baking; if the sugar is being used purely for flavouring, then you can use whatever sugar you like.  Much of the time in baking though, sugar is used for more than just sweetening, and it's important to be aware of that.  If the sugar is being creamed or dissolved, don't substitute unless you're sure you know what you're doing.