What parts of a bike could potentially be harmed by degreaser?
Degreaser acts like a solvent (although I'm unsure if it is technically a solvent, or more of an emulsifier).  With all solvents you'll have to be careful with what they contact--however degreaser is relatively safe compared to stronger solvents.

Each type of solvent has its own list of things it will dissolve--and even brand-to-brand can make a difference, so spot-testing is always recommended.

In general though, the risky things are:


Rubbers
Adhesives
Organics (leather, etc)
Porous materials
Paints
(obviously) Grease/lubricants


Your main concerns should be your tires, your paint, and (as you mentioned) components like grip-tape and seats.  Getting degreaser on these has the potential to stain--so extra precautions may be in order.  Typically your best option is to try and prevent it from contacting them, and cleaning it off quickly with mild soap &amp; water if you do make a mess (although soap and water itself will stain things such as leather if you're not careful).

A secondary concern is that you fully remove the solvent after it's done.  Leaving excess solvent/degreaser behind is a very bad idea--see Rubbing Alcohol below for tips.

As for other solvents/degreasers:


Rubbing Alcohol is fairly safe.  It has the risk of drying-out leather and rubber, but is very useful for cleaning parts--and especially useful for rinsing degreaser off of things such as chains (it also evaporates and leaves them dry--ideal for re-lubricating).
Naptha (a.k.a. liquid lighter fluid, not butane) can be very effective as well and is generally safe
Acetone is a stronger solvent, and likely to be less safe.  It should be used with extreme care, but is a very powerful solvent and can get the job done well and quickly.