What exactly is transient behavior?
Steady state behavior is behavior that continues on indefinitely, if you keep the system state and its inputs the same. 

For example, if you input a sine wave with amplitude \$a\$ to a linear system with gain 4, you'll get out a sine wave with amplitude \$4 a\$ and some phase offset. This behavior will continue as long as you continue to input the sine wave.

Transient behavior is any behavior that dies out over time.

For example if you stepped up the input of your linear system to an amplitude of \$2 a\$ at time \$t=0\$, you'd see the output jump in amplitude from \$4 a\$ to \$8 a\$, but it would take some time to do that. The behavior during that transition, from when it's behaving like it had a \$a\$ input for a long time to when it settles down to behave like it had a \$2 a\$ input for a long time, is the transient behavior.


  What does transient behavior look like, for example?


It depends on the circuit and the stimulus that produced it.


  Is it that slight upward curve when you zoom in really close on the graph of the voltage across the capacitor as you charge it?


It could be that. 


  Why does transient behavior happen? 


Because the steady state behavior is only a simplified description of the circuit behavior that describes what happens when the inputs remain the same for a long time. Transient behavior gives a more complete picture of the circuit behavior that also includes the possibility of inputs changing over time.


  Do we want it to happen or no?


It doesn't matter if we want it or not. It does happen, so as a designer you have to deal with it or design your circuit to produce the behavior you need.